bjlkeng

Comparison of PySpark vs Dask:

http://dask.pydata.org/en/latest/spark.html

One thing that bothers me with these articles is how they never address the software engineering challenges of writing large complex systems in a pure functional style. There are some tasks where just adding a sprinkle of imperative programming can make the design of the entire system much easier to understand (a kind of "mostly functional"). An article that really crystallized this point is here:

http://prog21.dadgum.com/54.html

Sysomos (a MarketWired company) - http://www.sysomos.com/ - Toronto, Canada

Sysomos is redefining social media analytics with a powerful product suite that provides customers with the tools to measure, monitor, understand and engage with the social media landscape. Sysomos analyzes huge amounts of social data from Twitter (full firehose), YouTube, Facebook, forums, blogs and many more.

We are looking for extremely bright individuals to join us as Data Scientists in the Toronto Labs team. We are a newly created team tasked to bring new applied research to the Sysomos social analytic platform. We use a whole bunch of techniques (stats modeling, ML, text mining, graph analysis etc.) to analyze the massive amounts of social data we ingest.

Requirements:

* Advanced knowledge of data mining, machine learning, text mining, NLP or information retrieval.

* Hands-on experience in statistical computing software (R, MatLab, SPSS), big data analytics tools (e.g. Hadoop, Mahout, Map-Reduce, Impala), or NLP packages (e.g. OpenNLP, LingPipe).

* Experience with database, search, or indexing technologies (e.g. MySQL, Lucene/Solr).

* Strong programming background with advanced knowledge of algorithms and data structures in a popular language (Java, C/C++, Python, etc.) in a Linux environment.

* Bonus points for implementation of big data/streaming data analytics or visualization, or working with large-scale social media data (e.g. Twitter, Facebook, Linkedin, Google+).

* Exceptional analytical, problem solving and communication skills are a must for close collaboration with colleagues and customers.

To apply contact me at bkeng@marketwired.com