SilverStraw This is definitely AMAZING!! As soon as I have time, I would like to modify my model to support hand tracking and try it out.
Spine Pro Vtuber Working Prototype
I updated and grouped the Spine animation names. The grouping is necessary as this project has grown to include various optional tracking modes.
https://itch.io/t/2951812/spine-animation-names#post-8031587
Standby Animations
- idle
- breathe
Face Animations
- face_roll_left
- face_roll_right
- face_pitch_down
- face_pitch_up
- face_yaw_left
- face_yaw_right
- left_brow_raise
- left_eye_open
- left_pupil_pitch_down
- left_pupil_pitch_up
- left_pupil_yaw_left
- left_pupil_yaw_right
- mouth_height
- mouth_width
- right_brow_raise
- right_eye_open
- right_pupil_pitch_down
- right_pupil_pitch_up
- right_pupil_yaw_left
- right_pupil_yaw_right
Body Animations
- torso_scale_y
- torso_scale_x
- torso_rotate
- left_upper_arm_scale
- left_upper_arm_rotate
- left_lower_arm_scale
- left_lower_arm_rotate
- right_upper_arm_scale
- right_upper_arm_rotate
- right_lower_arm_scale
- right_lower_arm_rotate
- left_upper_leg_scale
- left_upper_leg_rotate
- left_lower_leg_scale
- left_lower_leg_rotate
- right_upper_leg_scale
- right_upper_leg_rotate
- right_lower_leg_scale
- right_lower_leg_rotate
Hand Gesture Animations
- left_close_fist
- right_close_fist
- left_point_up
- right_point_up
- left_victory
- right_victory
- left_iloveyou
- right_iloveyou
- left_thumb_up
- right_thumb_up
- left_thumb_down
- right_thumb_down
Spine Vtuber Prototype 2.0.5
Optimized setting Spine animation for hand gestures only once. Prevent continuously setting Spine animation for the same hand gesture animation.
Added background thread support for body tracking AI on supported modern web browsers. The background thread start up the AI immediately without hindering the main thread. Users would experience shorter or no wait after starting the body tracking for the first time.
https://silverstraw.itch.io/spine-vtuber-prototype-2
https://silverstraw.itch.io/spine-vtube-test-model
SilverStraw Wow! It's amazing)
I am currently working on including a physics engine. Experimenting various ideas. There are physics constraints that have to be anchored to the Spine bones otherwise the physic bodies won't follow the Spine skeleton. There isn't any graphical user interface so that would have to be include before the release. Physics collision adds its own complexity and problems so I would have to add it at a later date.
I have a quick footage to show what I have been testing.
https://www.deviantart.com/silverstraw/art/Spine-Vtuber-Physic-Test-1-981902796
SilverStraw OMG, very cool!! It could be used in various ways!
Misaki Yes, several physic constraints could be used together to create various usages. In the video the ball is using gravity force that is pointing upward.
I'm going to try to support physics constraints for webgl runtime before official release.
https://silverstraw.itch.io/spine-vtuber-prototype-2
update 2.0.6
- Updated to run Spine 4.2 beta physics constraints. Include Spine 4.1 backward compatibility that 4.2 beta broke.
- Stopped drawing landmarks when face and body tracking modes under "AI Settings" menu are unchecked.
- Increased smoothing for body tracking because there is a lot more shaking from partial body than full body tracking.
- Fixed model number input that broke .svp saving process.
SilverStraw i'm almost done with porting the physics constraints. should be done today or tomorrow!
Patched HTML null bug and body tracking startup bug. Thank you, LustFire for notification.
https://silverstraw.itch.io/spine-vtuber-prototype-2
update 2.0.7
- Updated runtime to Spine 4.2.35 webGL with 4.1 binary backward support.
- Moved menu options to the right of the model display at the request of user for easier live stream changes.
- Fixed an attachment variable name bug that gave application errors.
update 2.0.7 amendment
- Fixed physics constraint bug that did not update correctly.
https://silverstraw.itch.io/spine-vtuber-prototype-2
update 2.0.8
The web application will switch between playing the eye blinking animation track and setting the track time based on tracking data. This should result in a smoother blinking.
Added a "blinking speed" property into the model's "Single Value Properties" to control the play back speed of blink. At default it should be "2" which is about half a second of blinking animation ( 1 second / 2 play speed ). Note that the duration will be shorter because the animation track will stop at where the eye tracking state. This animation stop is to prevent jumping from full eyes wide open to eye tracking state after the animation track finishes.
Hey @SilverStraw I've been following this thread for awhile now. Amazing stuff and very exciting to potentially take some market share away from Live2D (one of the most backward programs I've ever seen) Anyway I've done some of my own tests for the vtuber prototype and I've run into some problems. I'm not sure if I am the one that is the cause or if it's an issue on your end. I'm using the latest 2.0.8 prototype.
I took very simple square shape graphics and meshed it into the face, eyes, and mouth and I believe I did all the animation types that are necessary. Unfortunately the end result is not the same as my Spine editor file. The mesh goes crazy and stretches and pulls in directions that are in no way related to the file. Are there mesh limitation for the prototype or is it a bug that can be fixed?
If this can be fixed or if I can be informed on how to do this correctly without breaking it I'm confident I can make a very cool vtuber model with a combination of bones and mesh.
Lastly I just want to say thank you for building this prototype because the potential for this to be amazing is very high.
I've attached my spine file
Also here is a video of the issues I'm having
mikedssx Are you using physics constraints?
SilverStraw Yes! Are the physics constraints not ready for prime time yet?
Spine Vtuber Prototype
https://silverstraw.itch.io/spine-vtuber-prototype-2
Update 2.0.9
- Updated runtime to Spine 4.2.61 webGL. Spine 4.2 exports without back support.
Physics constraint should be fully supported now.