Shakalaka Boom Boom is an A.I. experiment which creates a magical experience by converting anything drawn on a mobile canvas into a real object in augmented reality world. The name of the project was inspired by an Indian TV series by the same name.
Check out the video below to get brief idea about what the project is all about!
I built this project along with my friends Arun and Kiran while participating in a hackathon. We just came up with this idea when wondering that if it's even possible to get what you are dreaming of by just drawing!
Then I built a Machine Learning model using TensorFlow to classify the drawings. I made use of the transfer learning technique in machine learning to come up with a quick model. A small data set was created for this purpose. Initially the model was trained to classify the drawings in five categories namely candle, chair, cup, lamp and vase.
On the other side Kiran built APIs to serve the model using Flask. And Arun built a nice iOS app using ARKit to deliver the final experience. Special shout out to Darshan Sonde for providing some valuable inputs while building the machine learning model and Cryssac Franson Aldo for designing the iOS app UI.
All the work we have done is open sourced. Go get your hands dirty by playing around with the code at Github.