Ask HN: Python vs Scala for a fast web-project

https://news.ycombinator.com/item?id=4077892

by ajakhu • 14 years ago

9 14 14 years ago

news.ycombinator.com

I was working on a web-project lately with a friend. We faced a situation where we realized that Python was being a bit slower than expected. Also, we were planning on using a database that was not native to Python (Python bindings used its REST interface).

So, we had to make a decision to either keep going in Python or port to Scala (Scala seemed apt for many reasons).

What would you have done in this case?

gelnior• 14 years ago

If you are comfortable with Scala and think it fits well for your project go for it. There are some frameworks like Lift or Play with which you can start quickly a web app.

ajakhu7• 14 years ago

Yes, we were planning for Scala/Lift only.

gaius2• 14 years ago

First profile the Python app - is it being slow in Python or is it waiting on something else? If it is being slow in Python, can you rewrite the critical section in C? And speaking of which, is there a C API to your DB that you can use from Python? The Python mentality has always been that it is easy to interface Python and C where necessary for performance.

pestaa• 14 years ago

Or better yet, have a look at Cython or PyPy. Both give a performance advantage over the standard intepreter.

ajakhu7• 14 years ago

The API problem will still exist even if we switch to PyPy or Cython.

ajakhu7• 14 years ago

No, there isn't a C API for the DB. So, that put C out of the picture for us.

gaius2• 14 years ago

Out of curiosity, may I ask which DB this is?

playing_colours• 14 years ago

Did you have any experience doing web development with Scala? I am now choosing stack for my new project and considered Scala/Play as an option. There are some problems both with Play2 framework atm(no Wax deployment support, long rebuilding after small changes, work with actions is confusing) and the language of Scala (complexity, version migration, SBT etc.) The situation is promised to get better in future but most need their work done now. If you are choosing bleeding edge, new shiny thing be ready to face additional problems which can slow your development. If you prefer just to get things done you'd review your Python code, find and fix bottlenecks. Maybe you can combine techs choosing what is appropriate for some work - say Python for web interface with critical parts in Java/Scala.

ajakhu7• 14 years ago

No, we didn't have any experience with Scala/Lift/Play. We were aware of the learning curve of scala. But what we thought was in the the best interest of project. I've read about the problems ingoing with Scala, but the benefits outweigh the problems most of the times. And yes, learning a new technology will definitely slow down the development (especially Scala).

manishsharan• 14 years ago

Its not the language but your framework and architecture and your own proficiency in the language ( whatever that might be) that matters most for a fast web application.

Your code would spend most of its time fetching stuff from DB, external web services etc and rendering HTML. Does your framework allow you to easily incorporate database caches? Does it allow to to cache frequently rendered HTML segments ? Can you offload certain tasks ( such as sending email etc.) to a secondary process via a job queue ?

You could write a web application in C that could still suck big time unless you get the application architecture right.

ajakhu7• 14 years ago

We had integrated redis in our project for server side caching. And I'd like to clarify that Python was being slow for regular cases (atleast not as fast as we wanted it to be).

As we were newbies to web development (although we have a lot of experience in development), we had put in our best knowledge while designing the architecture. We kept independent things independent. That's why we chose flask, it gave us the flexibility we wanted. We were free to choose the extensions we wanted to integrate.

petervandijck• 14 years ago

If your goal is to launch, stop fooling around with languages and rewriting your code. What exactly was slow?

ajakhu7• 14 years ago

1) A NoSQL database best suited our project. So, the non-nativeness was a hurdle. Neo4j bindings for Python used JPype, which was rather unmaintained. And I read at some places that it was a bad idea to mix python-java (especially using the unmaintained JPype).

2) Using Flask gave us so much flexibility as a framework. SQLAlchemy was probably the best thing that I've ever seen. But the work was made complex by the fact that Python was not able to give us the performance that we wanted. We tested the web-project running locally, and we noticed some exciting results. Even though we had done proper memory management from our side, the memory consumption was a little high than we had actually expected. I'm not an expert web-technologies, but as a programmer for last n years, I've learnt a lot about memory management. We projected that when the load would be actual, python performance would suffer.

3) We wanted a real-time performance application so that the user experience was the best. We implemented a part of the web-application and tested it. The results were unsatisfactory. Then we planned on shifting to scala.

Ask HN: Python vs Scala for a fast web-project

Related Stories