Should I consider a startup in 3D content creation, Metaverse?


First off the base: drawing is hard. Drawing in 2 dimensions is what we are used to do growing up. Pencil at hand, trying to sketch a dog — our dog. The first results? Oh well…

The Metaverse days

Today much has changed from my early days of 3D of the 1980s. 3D content is everywhere and it can even be used for full hollywood movies. In video-games it is so realistic that you get motion sickness! In fact Virtual Reality (VR) has also opened the door to more desire to create 3D scenes and environments, sculpting and arranging items as if we were in real life.

And it is only going to get more pervasive

What has not changed much is that creating this content is still hard, despite the enormous amount of code and tools we have today.

Drawing in 3D

Despite the great success of 3D, how many of you have ever actually tried to make a 3D drawing with a computer? Yes there are many tools available, like Blender, Cinema 4D, 3ds MAX, Sketchup, Maya, Mixamo, Houdini… and many more — but they all have a very steep learning curve, and unless you use them on a daily basis you will never be able to master even the simplest combination of capabilities and shortcuts they offer.

Drawing in 3D is hard

very hard… beyond comprehension if you have not tried. You basically have to specify the position of each and every 3D point in your meshes. Try to draw a real car for example one day, maybe using Blender.

Designing a 3D game, scene, level

A 3D item will just sit pretty on your computer if you do not use it. Maybe you want to make a game or a illustrate a scene or make a short powerpoint-like movie animation.

  • storytelling and scripting
  • design of 3D meshes, items, landscapes, skybox, etc
  • embellish meshes and assets with materials and textures
  • scene and levels composition
  • animation of some meshes
  • coding of interactions, game logic

Would you want to create a startup in the 3D space / Metaverse?

The issue remains: 3D design is hard and not open to the public yet. Young adults and kids would like to build games in Roblox, for example, but can’t because it is hard to design and hard to code. A tool that allows for easy scene creation and for simple dynamics is definitely needed!

  • 3D tools that are easy-to-use: tools like TinkerCAD are easy to use and should be the standard entry point to any 3D design tool aimed at the general public
  • Create 3D scenes out of existing assets: there are many websites colleting community-created 3D assets. These make the life of casual user easier, but do not provide a consistent look and feel for implementing an entire scene. These assets also mismatch in size, texture definition, polygons count. There is a need for a centralized database of 3D assets like Kenney.
  • Create 3D assets automatically: the Holy Grail is to be able to design an asset with just text: “a blue coffee mug that look like Lego”. A bit of a CLIP / DALL-E for 3D. We may be getting closer to this today, but the way is still long before we can provide consistent style, and fine-detail programmability that artists would want. Yet it may suffice for some casual users. One of the issue is that possibly specifying each 3D point with a neural network is overkill ad ill-posed, as there are much fewer control parameters in assets that one would want to vary.
  • Extend procedural 3D assets: the other way, that is between using fixed assets and neural network created assets, is to use procedural assets that are hierachical. These can be design with less parameters and even graphically (Geometry Nodes and Textures in Blender) and offer a hierarchy as a bonus. They may be just the perfect way to set up a large database.
  • Placing items in a scene: once you have all your assets for your scene, the next step is where to place them. Specify rules, script arrangements, or use a neural network placer?
  • Logic and animation: the ultimate step in 3D (and more) scene design is to be able to specify scene logic, interactions and animations purely with natural language. Today all this is done with computer code, but graphical and visual programming with blocks really helps (Scratch, Dreams, Garage Game Builder). Animations are more complicated and will require tools that can transfer them from a video to a character, or from a character to another character with different bone structure.

Who are you and your customer?

The first question if you area considering a startup in this space is who are you and who is your customer (as usual…).

What is out there?

The 3D design and artists tools environment is large and wide, and it is always expanding. Here we list some of the existing tools today.

Game making ecosystems

Unity and Unreal are the top game making frameworks, including all steps required to make a video-game or a digital 3D production. Needless to say they are a professional tool for professional coders and require entire university courses to master. Not useful for casual users!


Here are some reference I found useful to write this and survey the field:

AI research tools

GAN for graphics:

Making games

if you ever want to learn video game development this is one nice resource to start:

About the author

I have almost 20 years of experience in neural networks in both hardware and software (a rare combination). See about me here: Medium, webpage, Scholar, LinkedIn, and more…



I dream and build new technology

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store