SilverStraw I spent over a week ( working on this after my full-time work ) researching and developing a working Vtuber web solution. It's time for Spine Pro to level the playing field with Live2D in the Vtube realm. I expect to upload this application after I have enough features. Edit ( 7-4-2022 ) Spine Vtuber Prototype itch.io release The Vtuber Prototype using Esoteric Software Spine models is released. https://silverstraw.itch.io/spine-vtuber-prototype I also include the model that I have been using for testing. https://silverstraw.itch.io/spine-vtube-test-model
SilverStraw I beat you to the punch, Nate. :grinteeth: Nate a écritVery cool! We have plans to do this officially. It's still early days, but it's super cool! Official support would be awesome! :heart: In the meantime, I will spearhead the frontier. I bet your plate is full with Spine's future development. Maybe we can have some discussions in the future. 😉
Nate You did! :punch: Besides the camera/vtubing itself, we also have some supporting features in the works that makes vtubing super cool: physics! We're in a bit of a rut though, as we started these things and then switched gears to improve the new graph. Lately there's been a lot of distractions that eat our time and/or fracture our focus, plus the holidays, so even that's going much more slowly than we want. We do keep plodding along though, so we'll get there in time!
Erika Oooh now that looks very promising as well, SilverStraw! I'm so excited about being able to use Spine rigs for streams >.<
Mario Super cool! From the debug render on the left, it seems we are using the same tensorflow light model 🙂
SilverStraw Nate a écritYou did! :punch: Besides the camera/vtubing itself, we also have some supporting features in the works that makes vtubing super cool: physics! We're in a bit of a rut though, as we started these things and then switched gears to improve the new graph. Lately there's been a lot of distractions that eat our time and/or fracture our focus, plus the holidays, so even that's going much more slowly than we want. We do keep plodding along though, so we'll get there in time! Yay! Physics! It's been slow for me as well. Now, I am trying to figure out how to allow user to load their Spine model that would work with spine-ts runtime. Erika a écritOooh now that looks very promising as well, SilverStraw! I'm so excited about being able to use Spine rigs for streams >.< Hype! :fiesta: I can't wait to create Spine models for streaming. Live2D Cubism has a completely different approach to model rigging. Mario a écritSuper cool! From the debug render on the left, it seems we are using the same tensorflow light model 🙂 Are we really using the same? :bigeyed: Now, tell me more. :smirk: UPDATE: I finally got the web browser file reader to work properly with Spine WebGL runtime. My file browser isn't shown in this video but I am clicking the asset files to be read and loaded into Spine runtime. This is an important step towards Vtubing with Spine rigs.
SilverStraw Nate a écritCool! It can also be done via drag and drop. Working with browsers is such pain! Nate is telling me something. Okay, I found the example of the drag and drop from Spine Player generator example. I am updating on my progress with my application. The last couple months has been hectic with work and my other project failures.
SilverStraw Hello Mario and Misaki. This is another step closer. I tried several approaches to operate the mouth on the model.
Misaki OMG, it looks very easy to use! The mouth behavior looks natural 😃 Although the character has no eyelid yet, just being able to open and close the mouth conveys face expression quite well.
SilverStraw The character is evolving. I will get to the eyelids . . . eventually. This should be ready for the next phase of testing.
SilverStraw I used all my rare candies, Misaki. 😉 Spine Vtuber Prototype itch.io release The Vtuber Prototype using Esoteric Software Spine models is released. I also include the model that I have been using for testing. https://silverstraw.itch.io/
Misaki Congrats on the release! 😃 I tried with a simple character which was created based on your Spine project file and it looks good! :heart: Here's what I tried: Sorry about the lack of blink animations since I drew and rigged it in a hurry and about the closed mouth not being clean enough! I will make various modifications later. It may depend on the environment to do this, but it seems that eye pupil movement tends to be a bit jumpy. However, the response for facial tilt, direction, and mouth movements are very smooth! Also, the setup was easy to understand, so creating my own project was easy for me. It is a lot of fun to try out the movements with this tool 🙂
SilverStraw Eye glasses affect the reliability of eye pupil tracking.I think it's because of the glare on the eye glasses. I used another software called Vtube Studio and it also has this issue. Eye glasses are a bane to eye pupil tracking. Another factor is the small size of the eye pupils relative to the face. They were twitching a lot so I had to smooth the incoming signal.That solution isn't perfect. It's hard to distinguish jitter from eye pupil movements. In the future I will allow users to adjust these incoming signals.
Misaki I have added some modifications to my skeleton: Eye glasses affect the reliability of eye pupil tracking.I think it's because of the glare on the eye glasses. I used another software called Vtube Studio and it also has this issue. Eye glasses are a bane to eye pupil tracking. Unfortunately, I tried it without eye glasses last time and this time, but it seems the eye pupil tracking response is not very good :tear: Currently, the right eye and the left eye can register separate animations, but since it often happens that only one eye moves and looks weird, it would be good to make both eyes share the same animation once. Or, as in Owl's sample, it would be good to combine the animations for the face and eye direction together for now.
