Developing 3D for Cocos2d-x | Cocos2d-x Developer Blog

23rd April 2015 – Written by Peter Respondek

As of Cocos2d-x 3.0 we can now start adding 3D art to our games. This feature comes none too soon as the number of hand held devices out there that can render decent 3D are becoming far more prevalent. The past two years have seen a marked switch from 2D to 3D in high-end mobile game development and for many developers trying to find an edge in an ever crowded market, going 3D might be a way to get a leg up on the competition.

The video above shows a scene from my next game “Food of the Gods” running in Cocos2d-x 3.3 and was captured from my iPhone. In this article I’m going to cover what I did to make it and how to get it into the engine. For those familiar with the “Fantasy Warrior”, the Cocos2d-x demo, the main difference here are:

1. Light Mapping: You will notice everything is lit and has shadows. Lighting is as good as your 3D package can bake out which usually includes direct lighting, bounced lighting and shadows.
2. Vertex Blending: Notice that the road, grass and cliffs blend together without any visible seams.
3. Alpha Masks: Foliage are just plains with alpha texture.
4. Additive Billboards: For light beams and other atmospheric effects.

All of the assets are made using a 3D package called Modo and textured using Photoshop. I won’t go into too much detail about making 3D models or textures as there are a stack of tutorials floating around the internet, but I will provide extra details as they pertain to Cocos2d-x.

Meshes and Textures

As you can see from the picture below, all the models are made up of multiple tiling textures. The textures are generally 256 x 256 or smaller. You will notice that I bunched all my trim images onto one texture. This is an easy trick to reduce draw calls. The base textures are from http://www.cgtextures.com or around the web (i.e: Google Image’s search). I tile the images by using the old photoshop “offset filter and then paint over the seams with the heal brush” trick. I wanted a kind of painterly look to I used the cutout filter (note: the cutout filter also has the effect of making the png’s compress down real good) and then painting in extra highlights and shadows in some textures. I find straight photo source textures turn to noise when you reduce them in size.

Alternatively you could just make a unique texture for each mesh. This will work as long as the mesh is small, or the camera does not get too close to the mesh. If you are good at photoshop, it will produce better results too. Furthermore, because you are only using one texture you are only using one draw call. I would not do this for a building in a first person shooter though, because the texture would always be too low resolution when you get up close. My reason for not using a unique texture is because the process is very time consuming. The whole scene took me four days to make.

Light Maps

Once you have your mesh modeled and textured it’s time to bake out your light maps. Unfortunately Cocos2d-x doesn’t have any sort of light baking process like Unreal or Unity, but don’t despair, most 3D packages can bake out light maps for you and do a far better job than most game engines to boot. First, you will need to light your scene in your 3D package of choice and then make a second UV map for each mesh. Every surface of your mesh must be mapped within the 0 to 1 boundary of the UV plain. This sounds complicated and time consuming but in Modo its a very simple process. I just use the “Atlas map” UV tool and then use the “Pack UV” tool. These two commands automatically unwrap the mesh and gives you a fairly efficient layout.

Once you are done, set your renderer to only bake illumination and start rendering. You can also bake in some ambient occlusion if you want. You can get away with a fairly low resolution light map. It can arguably look better because the blurry pixel aliasing has added effect of making the shadows softer. These buildings above are all mapped onto one 512 x 512 light map. The whole scene has 4 x 512 light map textures. Make sure you leave a little bit of space between each UV island and set your renderer to expand the uv island edges a few pixels. This stops black borders appearing around the corners of your mesh as lower mip-maps kick in. That last bit sounded like a whole lot of 3D techno babble. For the benefit of those familiar with Texture Packer, the “Extrude” value does the exact same thing as what I just described. Takes the edge of the texture and smears it out so you don’t get those annoying gaps between sprites, which in this case will manifest as black lines around the edge of polygons.

If you want to improve performance at the cost of memory and package size you could bake both all the color and lighting information into one texture and avoid using a light map altogether. However, the texture size will need to be at least doubled to get a similar pixel density. It’s entirely up to you and the demands of the game you are making.

Next, lets talk about adding vertex color data. I apply vertex color on the terrain, which allows the shader to blend in the grass texture on top of the cliff without any visible seams. Any vertices that are painted white will have the texture you specify blended in. Actually I really only use the red channel for this example, but feasibly you could use all 4 channels to blend in different textures.

Finally, I broke the scene up into separate meshes. I have a separate mesh for each building, a mesh for the terrain, a mesh for the water, and a mesh for the alpha masked textures like the foliage and banners. I do this for two reasons. First, the terrain, buildings, water and alpha textures all use a different shaders. Second, we want to save performance by not rendering geometry outside the frustum of the camera. It’s important to note the the camera will use the bounding box of the mesh to determine what is visible and what isn’t, so try and break meshes into chunks that have small bounds.

Exporting

Once your done modeling and texturing we need to save out an .fbx file for each one of our meshes. Fortunately most 3D packages support fix. Autodesk freely supplies an SDK for the format. Unfortunately, Modo 701 has rather error prone exporter for fbx. I had to write my own export script to get both the second set texture coordinates and vertex color to export correctly. You can download the exporter from the “Modo Scripts” section of my website.

Once saved you will need to use the fbx-conv.exe command line app that comes with Cocos2d-x. It should be in the /tools directory of your Cocos2d-x root.

The “-a” exports both the binary (.c3b) and text formats (.c3t) of the mesh. The text format is useful to see if everything exported correctly but don’t put it in your resource directory. You should hopefully see the following at the start of your c3t file if everything exported correctly:

Pay attention to that VERTEX_ATTRIB_TEX_COORD1 attribute. If it’s not there your light maps won’t work. If your are exporting a mesh with vertex color you should see an attribute for that also. It is also important that the texture coordinates are in the correct order. I always use the first tex_coord for the tiling textures and the last tex_coord value for the light map. For those using Modo, uv maps are ordered alphabetically.

Shaders

For the longest time I had a lot of trouble understanding GLSL and shaders, but like most things in programming at some point it just clicked… followed by the distinct feeling of being a complete idiot. Shaders are actually really simple once you know whats going on. If you want to do anything beyond putting a flat texture on a mesh in Cocos2d-x you will need to learn how to write shaders. Cocos2d-x does not have any fancy visual shader editor like Unreal at the moment, so there is no getting around it.

In this section I’m going to go through the shaders I wrote for my scene and explain what I did and why. If are already familiar with shaders you can just skip over this bit.

First lets look at applying the shader onto your mesh.

https://gist.github.com/Rahnem/5b642b55910071a7d5b4

This is ripped from the Cocos2d-x cpp-tests project. You will want to break it into functions to avoid redundancy if you are loading a lot of meshes with different shaders. For right now lets just concern ourselves with the lines below and have a look at the shaders:

https://gist.github.com/Rahnem/412fbba9492624872601

The “lightmap1.vert” is the vertex shader. If you mesh has a shader applied to it then every vertex will get this operation run on it every frame. Likewise, “lightmap2.frag” is the fragment shader and every pixel of the texture on your mesh will get the operation applied to it every frame. I’m not sure why they decided to call it a fragment shader, I always think of it as a “pixel” shader. It’s easy to see why lots of shader instructions, especially in the fragment shader, can lead to bad frame rate.

Lets break down the vertex shader in detail:

Attributes are supplied by the renderer. “a_position” is the the position of the vertex. “a_texCoord” and “a_texCoord1” correspond to your two uv maps. Remember “VERTEX_ATTRIB_TEX_COORD” at the start of your .cbt mesh file? These are the values that those attributes link up with. You can grab many other attributes from the render including the vertex normal and vertex color. For a full list of attributes take a look at CCGLProgram.cpp.

“varying” values will get sent to the fragment shader. Any variable that the fragment shader needs to know about should have the “varying” qualifier. In this case we just need to know about the two texture coordinates.

Set the vertex position and copy the texture coordinates into the varying values so the fragment shader can use them. Now, lets break down the fragment shader.

Declare the “varying” values that are passed in from the vertex shader.

Remember that ‘state->setUniformTexture(“lightmap“,light map);’ line when you applied the shader to your mesh? This value corresponds to the texture you sent in that statement.

This statement sets the pixel color. First you will notice the “CC_Texture0” variable that was never declared. Cocos2d-x has a number of default uniform values that you can use in your shaders. Again, refer to CCGLProgram.cpp for a list of these values. In this case CC_Texture0 refers to the texture you applied to your mesh in your 3D modeling program. The texture2D command looks up the pixel color and alpha of a texture at a given texture coordinate. It returns a vec4 with the r,g,b,a value of that pixel. So on this line I look up the color of my tiling texture in UV 1 and then the color of my light map texture in UV2 and multiply them to together.

You will note that I first multiply my light map color by two. Color values texture will only ever range from 0.0 to 1.0. So obviously if you multiply vec4( 0.5, 0.5, 0.5,1.0 ), which is a neutral grey, by vec4( 1.0, 1.0, 1.0, 1.0), which is full white, you will still get vec4( 0.5, 0.5, 0.5,1.0 ), neutral grey. Multiplying by two allows the light map to brighten the as well as darken the tiling texture color giving you a nice range of brightness.

That is shaders in a nutshell. For further info on shaders I suggest reading to following: https://www.khronos.org/opengles/sdk/docs/reference_cards/OpenGL-ES-2_0-Reference-card.pdf

Now I’m going to show you examples of the other shaders I used in the scene:

https://gist.github.com/Rahnem/93e2efa4b4e5cf2ec1de

This fragment shader uses the same vertex shader as before and does the same light map calculation, but this time discards any pixel with an alpha value less than 0.5. If you try to add a alpha masked texture without this shader you will get all number of sorting issues. All the plants, banners and leaves in my scene use this shader.

https://gist.github.com/Rahnem/5305bca63de43599c10a

This is the shader for vertex blending which I use blend the grass texture into the cliff top. You will notice the “a_color” attribute in the vertex shader which corresponds to our vertex color. Here I use the mix command to blend the cliff texture with the passed in grass texture using the red channel of the vertex color as the blend amount.

You could achieve the same effect by using another texture for the blend mask, however using vertex colors use less memory, have a smoother gradient and is more forgiving when you want to make changes to your mesh.

https://gist.github.com/Rahnem/c7abb7b79062aade0cf2

Currently, the scene is a little bit static and lifeless. It would be nice if we could add a little bit of movement to the banners and trees to make them look like they’re blowing in the breeze. Surprisingly its really to easy find a wave motion shader with about a thousand instructions but hard to find one with only ten, so I made my own. In order to make this work you will need to pass in the “u_time” value each frame:

The shader pushes each vertex of the mesh along it’s associated vertex normals in a wave motion based on it’s Y position. Basically, it gives the mesh a wavy motion making it look like it’s moving with the breeze without using any complex physics calculation.

Pretty easy huh? You can do all sorts of things with shaders and now that I have finally wrapped my tiny little brain around them I’m excited to try out different possibilities.

Billboards

Now that I have all my meshes and textures made, shaders written, everything is imported and running, it’s time to add a little juice to the scene. Using Billboard and Particle objects you can add a lot of atmospheric effects like light rays, dust, sun glints, haze, ground fog etc. In the video you can see the light rays streaming down onto the scene. These light rays are just big billboards the the additive blend func turn on. The fires are billboards with a texture animation I made in modo.

Troubleshooting

So far I have made this process seem fairly painless but no doubt you will run into trouble trying to get your meshes into the engine and getting them to look right. Most of my problems were fixed by examining the .c3t mesh file and figuring out what was not getting exported, or getting exported out of order. Each set of attributes section in the .c3t file corresponds to values in the vertex section. This is handy to know if your uv maps are getting exported out of order. Light maps will alway between a value of 1 and 0 while tiling texture can be any number.

If you see a red texture instead of the one you wanted it means Cocos2d-x could not find the texture you assigned. Check the bottom of the .c3t file where all the material paths are and make sure they are pointing to the correct location. If you can’t see your mesh at all make sure the camera is looking at it and the far plane is large enough to see it.

Automation

After a few days I found adding things manually into the scene, hooking up all the shader attributes and placing billboards became extremely tedious. Writing a script to convert all you meshes to c3b is easy enough, however, with no Cocos 3D editor at the moment, adding items in to a scene can be a really hit or miss affair because you have no way to visualize the 3D position of your objects. My solution was to write a script for modo that would export all the meshes in my scene from modo and writes a .plist file that contains all the data about the scene. Python already has a library for writing out plist files call “plistlib” which was very convenient. I then parse the .plist in Cocos2d-x to build my scene. I used locators in Modo to mark out billboard locations and customs channels to pass variables for things like what shader to use or what light map to use etc. It’s not a great solution but it’s good enough for my needs.

About the Author

Peter Respondek has been a designer and 3D artist in the AAA game development industry for the past 10 years. He has worked for many well know companies including id software (Rage) and Epic Games (Unreal Tournament 2004). Recently he has been trying his hand at programming and solo mobile game development using Cocos2d-x. “Clear For Action”, his first game independently developed game, is available for iOS and Android. http://www.prespondek.com