I'm hoping for an artificial hand with a new simulation

I'm hoping for an artificial hand with a new simulation

The human hand is one of the most amazing and complicated parts of the body, capable of exerting both brute force and delicate manipulation as needed. Despite decades of research, scientists know little about the underlying structure and how the muscles and tendons work to move the hand's many bones in relation to one another. Without knowing how a real hand is constructed, it is almost impossible to build a model that replicates its anatomy and movements. This lack of insider information is why creating a computer simulation of the work of a human hand is one of the most difficult problems in the world of computer graphics and animation in particular.

belushi | Shutterstock

Now that's what a new study saysHand modeling and simulation using stabilized magnetic resonance imagingreported at ACM SIGGRAPH shows a simulation that includes not only the skin, but also the muscles, bones, tendons and joints.

The hand is very complicated, but before this work, no one had created an accurate computational model of how anatomical structures inside the hand actually move when it is articulated.”

Researcher Jernej Barbic

The expertly detailed model could advance the development of an artificial hand and could also be crucial for training a new generation of medical and paramedical students, building robotic hands and simulations for virtual reality training models and games.

How they did it

The first step was to create a team of computer animation experts and those skilled in creating simulations based on physical reality, as well as radiologists and other anatomical specialists.

The next challenge was to find the right imaging method that could systematically capture details of the hand's anatomy at every step of its movement. MRI scans provide a wealth of detailed information about the anatomy of the hand, but require the hand to remain completely motionless in each pose for about 10 minutes - which is not realistically achievable.

Barbic says:"It's virtually impossible to hold your hand steady in a fixed position for 10 minutes. A fist is easier to hold still, but try half-closing your hand and you'll find that after about a minute or two you start to shake. You can't hold it still for 10 minutes."

Making a support form

Therefore, to achieve this, they set up a production process to keep the hand stable in each pose, using materials from the field of special effects. In lifecasting, the human form is first formed and then rebuilt using plastic, silicone or other materials. Barbic found a cheap and readily available tool for cloning a human hand at a visual effects store. Barbic says of his find: “That was the aha moment.”

The third step was to create a plastic cast of the hand they wanted to depict, showing every minute detail, including the pores and tiny lines on the surface of the skin. They built a life cast out of an elastic rubber material, creating a 3D negative mold that could ergonomically hold the real hand in the required position for as long as it took to complete the MRI scan. Now 10-minute scans of the hand were taken in a different position on a male and a female model. There were 120 scans in total.

Understanding bone movements

For each pose, the scientists cut the entire hand into equal segments, called bone nets, according to the animator's network of connected vertices and triangles. These help show how individual bones changed position in each pose. In the end, the scientists were able to describe the exact musculoskeletal system in action for each hand position. This was fundamental to create a precise bone graft based on interpolative and extrapolative MRI-based data for all bone networks.

Creating the moving animation

This led to the final step: constructing a motion simulation that allows modeling any possible hand posture using the underlying skeletal motion data, including complex rotations and translations of individual bones during different types of hand movements.

The soft tissue simulation was then created using a method called FEM (Finite Element Method) to incorporate the calculated motion of the hand's muscles, tendons and associated fatty tissue as expected from skeletal motion. They introduced modifications that enable stable and faithful representation of skin folds and creases during joint movements. Finally, they added the surface details, culminating in a smoothly moving animated hand that can assume any position, even one that isn't part of the original set.

Value of this simulation

Of course, the work will be extremely valuable for those who design and produce computer games and films based on computer-generated imagery (CGI).

This is currently the most accurate hand animation model available and the first to combine laser scanning of hand surface features with an underlying bone manipulation model based on MRI.” Barbic adds, “Understanding the movement of the hand’s internal anatomy opens the door to biologically inspired robotic hands that look and behave like real hands.”

Co-researcher George Matcuk

In the next step, the researchers want to make their MRI data publicly available and add many more poses recorded on a total of ten models over a period of three years. This will help simulate and ultimately recreate the human hand. It could also be used to reach medical students who need to understand how the hand moves and its structure. According to Matcuk,“As we refine this work, I think this could be an excellent teaching tool for my students and other physicians who need an understanding of the complex anatomy and biomechanics of the hand.”

The team also wants to improve the model's sensitivity to muscle and tendon movements so that it can respond to actual movements in real time, as opposed to the current hour-long calculation process for a minute-long simulation. Its goal is to increase the speed of data retrieval and data calculation without compromising the quality of the simulation.

Sources: