The challenges of Creating a Civilization Web Game

http://code.google.com/p/freeciv-forever/wiki/FreecivWebClientStatusAndChallenges

by roschdal • 17 years ago

81 74 17 years ago

code.google.com

lionheart• 17 years ago

Well, the first thing I would suggest is putting all of the images into one file then using CSS to control what portion appears.

That should cut down on both latency and issues with IE only downloading one image at a time.

roschdal4• 17 years ago

So you are suggesting to stop using the HTML canvas element, and instead use Javascript to render tiles which are clipped using CSS from one large image?

sjs3822• 17 years ago

I think that was a response to the "How to handle a graphics tileset which consists of many 96x48px tiles?" question.

lars• 17 years ago

I think you could do both. Position 96x48px divs with css sprites over the canvas element to draw the images. I think this would be a performance win too, because browsers are already very good at drawing things like css sprites (although I don't know how well it combines with the canvas element).

sjs3822• 17 years ago

This technique is often called "css sprites" if the author wants to read more about the technique.

wmf3• 17 years ago

A somewhat related idea: Shouldn't it be possible to implement "canvas sprites" (no CSS involved) by downloading a large image, carving it on the client side, and drawing the sprites into a canvas?

patio11• 17 years ago

Several hundred CSS sprites in a file is pushing it. It would probably be a better idea to keep it to a ~20 or so per file. Plus, if I remember playing Civ, you can lazy load the suckers -- when you start out all you can see is a lot of black, a settler, and maybe 6 or so unique sets of tiles, plus the game interface. You don't need to block on waiting for the battleship icon for quite a while.

Civ also has some features which make it very easy to cheat creatively regarding logic for loading those things. You can guarantee, for example, that no Riflemen will show up anywhere on the map until someone has discovered Gunpowder. Thus, if someone discoverse Gunpowder, you send everyone an <img src=".../gunpowder_units.gif" style="display: hidden;" /> and you'll have the Riflemen sprite available instantly once someone actually constructs one. (Edit: There are, of course, security issues associated with that example.)

Internet Explorer has a limitation where only one image can be downloaded at a time

Really? Are they sure they don't mean "IE only downloads two resources in parallel from any particular domain, like suggested in the HTTP spec"? This has a fairly simple solution, familiar to anyone who has read the YSlow stuff: create a few subdomains to alias your HTTP server, load images from a few of them at a time. I do it for my website, works like a treat.

nostrademons8• 17 years ago

I would sprite each set of tiles, based on the dimensions. So you'd have one image for all the terrain tiles, one for all the terrain decorations (grains/shields/whales/fish/game/etc.), one for all the improvements, one for city styles at each game era, one for units (again based on game era - put your phalanxes/warriors/settlers in one sprite, your battleships/airplanes/carriers in another), etc.

This also avoids some nasty binpacking problems around combining sprites of different sizes.

When the game starts up, you only need to preload the terrain, terrain decoration, and initial unit sprites - that's probably a couple dozen, no more than you'd need to draw the page as an image. Less, actually, since you can re-use your terrain sprites.

You should then be preloading the other sprites as the game goes on. If the user isn't doing anything, use that network connection to download the next spriteset the game will need.

cool-RR• 17 years ago

What's the reason for not doing it in Flash?

krakensden• 17 years ago

Insufficiently trendy.

roschdal4• 17 years ago

I might have to investigate Flash further now. Firstly, at the time I started the project, there was no Flash plug-in available for my OS (64-bit Linux). Secondly, I would have preferred using only open standards for this open source project. Finally, and most imporantly, I didn't know Flash or Actionscript.

zvikara• 17 years ago

You may also consider using Silverlight. The .Net framework on the client is more powerful and capable than Adobe offering AFAIK.

sachinag• 17 years ago

Unless the purpose is to make this into some ginormous company, I'd highly recommend excluding IE from your list of worries. If it's a love-of-the-game project, do it to your (and your friends') satisfaction.

a-priori2• 17 years ago

I recently looked into using Flex to develop a Flash interface, and found it quite simple to get started... just beware of problems with asynchrony (that's what tripped me up the most).

wmf3• 17 years ago

Speaking of openness, if there was a large open source game written in Flash (such as FreeCiv), the swfdec/gnash developers might be willing to invest the effort to be able to run it.

mynameishere• 17 years ago

Also, Java applets are due for a revival.

DrJokepu3• 17 years ago

My solution: drop support for IE. No, seriously. I understand that it has a very large "marketshare" (installation base) but I think that very often IE is used on office desktops, that is places people wouldn't use for playing games anyway.

While there were times when IE was a decent browser it is clearly standing in the way of innovation at the moment. It is very easy to download a modern alternate browser and a large number of websites using the canvas element would force Microsoft to build a fairly good implementation of it into IE 9 (as it was I believe hinted by the IE project team leader anyway).

mcav• 17 years ago

By the time you finish and have it running smoothly, IE9 will be closer to completion (if not already released). Then you already have support for IE.

So unless there's a burning desire to market this to IE users right this minute, save yourself the hassle and let IE catch up with standards while you work on a better javascript client with Canvas.

staunch• 17 years ago

> ...IE is used on office desktops, that is places people wouldn't use for playing games anyway.

Pull the other one!

nostrademons8• 17 years ago

My startup was based around Flash-gaming, and our traffic numbers (to the extent that we had traffic, which wasn't enough to keep going but was enough to see some patterns) showed that the vast majority of hits occurred between 9 AM - 6 PM Eastern, Mon-Fri. Tailed off until about 8 PM (my guess is those were the west-coasters), then we'd have very few hits at night or on the weekend. Funny how that works. ;-)

cakesy• 17 years ago

OK some games, but I am yet to see anyone playing Civ 2 during work hours, or Alpha Centauri. That would sure make me laugh.

jasonkester3• 17 years ago

If I were them, I'd ditch the project now. IE support is not their problem. Everything they're doing just screams "this is my first try at game development", as well as "this is my first try at DHTML", and they've bitten off more than they can chew.

As it turns out, IE's rendering engine is actually faster than Firefox's. I know this from having written a pretty heavy Canvas-based web application recently, and from a dozen years of writing video games for browsers. If he were to ditch FF support, he'd actually have a better shot of getting this thing working.

But really, he doesn't have a chance, so the point is moot.

webwright• 17 years ago

Seriously? You're hanging out at news.ycombinator.com and claiming that he should quit developing something because he doesn't have prior experience?

sstrudeau• 17 years ago

I agree with the suggestion to drop IE. If someone is willing to dedicate enough time to play through a Civ game, they're probably not at work and probably willing to install a third party browser, so you're not sacrificing too much potential "market share," and saving yourself some grief; assuming Canvas performance is sufficient on FF3/Safari4.

Some other suggestions:

* Investigate multipart XHR

http://blog.digg.com/?p=621

This is "bleeding edge" and may not, in the end, work, but it'd be worth looking at the work Digg has done w/ multipart XmlHttpRequests (cramming multiple objects/files into a single XHR). The demo they've posted works pretty well in non-IE browsers and it looks like they've laid the foundations for supporting COMET-like delivery of objects over a single request, and have thought about how to address client side caching of objects.

* Business model? Is this just an OSS project or an attempt to build a business (or both)? I could imagine a small business that might generate enough $ to pay some coder/admins & some freelance graphic artists to work on the open source part of the code by providing easy-to-set-up on demand game servers for multi-player FreeCiv games, perhaps w/ a sister set of web apps for collecting game stats/organizing games/etc.

Good luck!

nostrademons8• 17 years ago

Canvas is also too slow on FF/Linux. Actually, for much of the Canvas stuff I've done, the bottleneck has been Firefox, not IE. excanvas.js and VRML isn't that slow; it's no speed demon, but Canvas is just about as slow on FF/Win and much slower on FF/Linux.

mofey2• 17 years ago

You may want to look into how Quake Live is delivered over the web..

pistoriusp2• 17 years ago

Quake Live uses a plugin. It doesn't "run in the browser" like one might expect something to "run in a browser."

mofey2• 17 years ago

That was part of my point. Instead of trying to do in something that was not meant for it, consider an alternative way.

wmf3• 17 years ago

In other words, compile the FreeCiv client for NaCl.

mahmud• 17 years ago

Unite3D delivers a kickass plugin for games.

http://unity3d.com/

But it's a commercial product and might be too expensive for the OpenCiv guys.

dood• 17 years ago

I suggest dropping the project, since civ has cost humanity enough already - browser civ could cripple my productivity... just a few keystrokes away when already procrastinating. I both love and fear this project.

dave_au• 17 years ago

I had to write a freeciv patch for my girlfriend which adds a daily move and/or time limit to the game.

It was much appreciated, still have a couple of odd behaviours that I'd have to iron out before letting it out into the wild.

pohl2• 17 years ago

This is a cool project idea. My first impression is that this feels like a natural fit for GWT. The ImageBundle mechanism would allow you to automatically build css sprites at compile-time -- and the logic for accessing the individual tiles from the sprites: (The gwt-canvas project is a small wrapper that would allow you to draw instances of the Image class to the canvas. http://code.google.com/p/gwt-canvas/ )

It would also make it easier to move larger chunks of the FreeCiv client into the browser in a manner that would be easier to refactor and maintain than Javascript - and while being compiled to be as small as possible.

Java, for all its faults, is strong in refactorability, which would be essential for keeping the code in sync with a moving target like the current C implementation.

Keyframe• 17 years ago

My first thought too - this also might be a perfect fit for something like vaadin where you do everything on the server side and push it to the client gwt for rendering.

radu_floricica3• 17 years ago

Well, with this architecture it'd be a real surprise if it worked. It's a brave attempt, but it was a long shot from the beginning.

I wonder how it would work as a java applet. Might require a jre download, but it should be many orders of magnitude faster then javascript.

cellis6• 17 years ago

Well if we're going to bring out the big guns, the plugins, this would be a different situation entirely. Besides, if a user can download a jre, why not just download the whole game in C?

radu_floricica3• 17 years ago

I'm not sure you need a jre. If I were to guess I'd say no, but it's been awhile since I played with applets on computers who didn't have it installed.

But web games are better for a host of other reasons:

- compatibility between os-es

- compatibility between versions of the same OS (win 95 vs xp) - much greater lifetime

- no hassle with installs, play instantly and from anywhere

- easier to multiplay

- switch from "savegames" to a "profile".

Maro• 17 years ago

Can you do some kind of differential download and local move? Eg. when the user moves his mouse to the left 20 pixels and the world scrolls, then you only have to discard 20xheight pixels on the left and fetch 20xheight pixels from the server to bring in on the right and move over the entire canvas (or whatever) by 20 pixels, without calling setpixel() for each moved but unchanged pixel. Then add any other fluff such as a building exploding or whatever.

nostrademons8• 17 years ago

I think that'd actually be slower if it means you have to do the drawing on JavaScript (vs. the browser's native rendering engine). As a rule of thumb, anything on an interpreted JS engine (not Chrome or FF3.5) runs about 1000 times slower than the equivalent C.

The way I'd do it is create a bunch of large tiles (like the size of Google Maps) and pre-draw all the sprites onto a single tile. Put those tiles into a single container div that is bigger than its container, positioned absolutely, and clipped with overflow:hidden. When the world scrolls, simply change the positioning of the container. If it scrolls far enough that you need to render another tile, drop the one off-screen and pre-render its replacement on the other side, so if the user scrolls even more, the images are waiting for them. Much like how G!Maps does it.

This offloads the graphics work onto the C++ rendering engine, which has been tuned for this sort of stuff. Doing it in JS is just asking for trouble.

pyre• 17 years ago

That might only work for the map though. I wouldn't want to have to download a pre-rendered tile every time that I moved a unit. It would make more sense to just pre-render the map tiles, and possibly the city tiles. Then the units would still have to be downloaded to the client and drawn on top of the map.

The same goes for changes that occur to the map (at least changes within the current view). Map tile improvements would be a pain if you had to download a section of the map just to update a single tile. It would probably make sense to have the sprites on-hand to update the currently viewed map section, and have it automatically added to future pre-rendered tile updates. This way you wouldn't have to download the map you already have to see the improvement, but when you downloaded future sections (or even come back to the current section) it will be part of the pre-rendered section.

All of this is disregarding whether or not there is a mode to show the map without improvements on it (IIRC, there was an option like this in Civ2). Then you might have to change these design decisions to accommodate that unless you're ok with the user needed to re-download all map sections whenever that options is flipped on/off.

mryall4• 17 years ago

If redraw speed is a problem, perhaps in-browser vector graphics (SVG, VML) would be a better solution than raster (Canvas).

There are probably a few libraries around for this sort of thing, but the one I've had experience with is Raphael: http://raphaeljs.com/

It should mean less time redrawing the map because you can just reposition the elements on the screen dynamically.

jasonkester3• 17 years ago

Never use a 3rd party library to do your core functionality.

This guy's whole project is going to revolve around DOM and Canvas stuff in Javascript. He needs complete control over that, so relying on somebody else's library for it will kill him.

If you're building a business app and need graphics for reporting, then sure, use a library. Use jQuery even. But if you're building a video game, you're going to need to do this stuff yourself.

mryall4• 17 years ago

The author already mentioned that he uses ExCanvas for IE. I don't think it's a stretch to use a library for what would otherwise be tedious recoding of the same logic in both SVG and VML.

If a library helps you deliver the software faster and better, use it. If it gets in your way, then consider using the layer underneath directly.

But, regardless, my main point was to consider browser-based _vector_ graphics, which are several orders of magnitude faster than canvas for activities like this.

cellis6• 17 years ago

I'm going to go with no. Vector graphics are only feasible as long as the shape bytes it would take to represent an image with nodes is less than using pixels. Because many of the items in his games have gradients, and depth, I think vector art would take a lot more space than simply using pngs. Of course this is only dealing with latency; I'm not quite sure of the vector vs. raster calculation tradeoffs but i can't imagine it being very much in vector's favor with relatively few calculations per frame.

cellis6• 17 years ago

Edit: also because you are using an XML format with SVG, you'll need to be conscious of parse-time (vs. binary format which is handled natively). This is non-trivial (trust me, i've tried it). Most game developers don't think like web developers: optimization is at the forefront of the development, or should be, or else you're going to have major problems with playability.

mryall4• 17 years ago

Fortunately, SVG and VML don't limit you to just vector graphics. You can render, transform and scale raster graphics within the vector graphics space. The graphic is just like a textured polygon that you can move around.

Consider how 3D engines work, where they compute all the visible surfaces using vectors, and then transform and scale raster "textures" to apply on these surfaces.

The XML issue isn't actually relevant either because normally the SVG/VML DOM objects will be generated at runtime by JavaScript, rather than serialized in the page and served up statically.

GavinB• 17 years ago

I think the real question is, where will you possibly get sufficient cleavage to advertise it? http://www.codinghorror.com/blog/archives/001286.html

wlievens2• 17 years ago

God I hate those ads. They pop up everywhere. I hope that game dies.

TweedHeads• 17 years ago

Try these options:

1. Go with SVG and WebSockets.

2. Go with pure HTML,Javascript and CSS with transformations.

I am working right now on a game based on SVG and couldn't be happier. Works flawlessly on FX, WK & OP.

One last advice, drop support for IE and save headaches and wasted resources.

nostrademons8• 17 years ago

I've put most of my comments as inline responses to other people, but here's one more:

If AJAX is too slow, do not think Comet will solve your problems. It's very hard to get Comet to work reliably on all browsers across all types of firewalls and proxy servers.

I did a prototype of something using the library that powers GTalk, Google Docs, and now Wave. Even with the library all pre-built for me, it was pretty hard to keep it from occasionally dropping connections and associated messages, and once in a while it'd lock up my browser (much like GMail and Docs do, occasionally. ;-)). I also peeked under the covers, and there's a lot of subtlety in what's going on.

Comet is best for when you have stuff that really needs to be real-time. Things like stock quotes, or chat. IIRC, everything in Civilization is done in response to user input. AJAX is fine for that.

Tichy• 17 years ago

I don't understand why he is trying to use canvas, rather than positioning images with HTML and CSS.

I haven't coded something like that myself, but I have been wondering if CSS layers would work nicely for isometric graphics.

I am surprised how many people suggest Flash. Flash needs to die, so it is a worthy goal to try to implement games in Javascript instead. Flash simply is not available anywhere and causes a lot of suffering.

The challenges of Creating a Civilization Web Game

Related Stories