dmitrykoval

This is really cool and promising product! So many times when I was getting a google docs spreadsheet from non-engineering folks I was wondering if only I can embed a small jupyter notebook with a few python cells and a nice looking pandas chart, using this data.. Congrats on the launch!

Sure, I agree, above certain usage threshold hosted Trino becomes totally justified. But then, some engineering time to maintain the cluster has to be factored in as well.. for anything ad-hoc in nature, I would start with Athena by default.

other arguments aside .. Athena costs $5 per 1TB scanned and also supports predicates pushdown to S3 Select. I wouldn't call this expensive, at least in comparison to self hosted Presto.

For this particular example direct initialization i.e. std::string product("not worked"); would be preferred, as you end up using one call to constructor instead of two: default constructor followed by the move assignment.

https://en.cppreference.com/w/cpp/language/direct_initializa... https://en.cppreference.com/w/cpp/language/move_assignment

Both are legal statements.

The second one is a direct initialization, invoking corresponding constructor.

The first one first invokes default constructor and then copy or move assignment depending on rvalueness of the arg.

Good progress overall, especially on the Rust side, I wonder if C++ and Rust would at some point follow the same roadmap when it comes higher-level compute features or rather deviate and develop at their own pace.

Special kudos to the Rust team for Parquet predicates pushdown feature.

That's correct, but SQLite would require to serialize/deserialize the data sent to Python func (from C to Python and back), while Arrow allows to get a "view" of the same data without making a copy. Which is probably not an issue in OLTP workloads, but may become more visible in OLAP.

Following similar observations I was wondering if one can actually execute SQL queries inside of a Python process with the access to native Python functions and Numpy as UDFs. Thanks to Apache Arrow one can essentially combine DataFrame API with SQL within data analysis workflows, without the need to copy the data and write operators in a mix of C++ and Python, all within the confines of the same Python process.

So I implemented Vinum, which allows to execute queries which may invoke Numpy or Python functions as UDFs available to the interpreter. For example: "SELECT value, np.log(value) FROM t WHERE ..".

https://github.com/dmitrykoval/vinum

Finally, DuckDB makes a great progress integrating pandas dataframes into the API, with UDFs support coming soon. I would certainly recommend giving it a shot for OLAP workflows.

Story by TC - https://techcrunch.com/2021/04/30/what3words-legal-threat-wh...

Architecture of a Database System – Hellerstein, Stonebraker & Hamilton, 2007. Is a great paper focused on DBs design and architecture. From 2007 but still rocks. http://db.cs.berkeley.edu/papers/fntdb07-architecture.pdf

That's one of the core use cases for Vinum - to provide SQL query engine, complementing Pandas in data analysis (thanks to Arrow), yet retain an ability to use native python and numpy funcs as UDFs. Also, it doesn't require input dataset to fit into memory.

https://github.com/dmitrykoval/vinum

It's a little bit strange that even though Python was mentioned there's not a hint about the actual problem in question.

HERE | Senior Software Engineer | 60-75k EUR | Berlin, Germany | Full-Time | ONSITE | VISA

My team, HERE Datalens is looking for great backend engineers to help us crunch and visualize big geospatial datasets. More about our product here: https://goo.gl/gcj8h3. About HERE in general: http://here.com/

We are looking for strong backend engineers, ideally with experience in crunching big datasets. Exposure to GIS/Maps is a big plus. We mostly use python in our stack, but depending on the task may switch to Java or C++, so we don’t expect python expertise, but you should be proficient in at least one of modern languages/ecosystems. Good grasp of algorithms and ability to reason about backend architectures is required.

Our team is relatively small and we try to operate as a startup. This is a full-time position based in Berlin and we do provide relocation package for those willing to move here.

This is a full position description on StackOverflow https://goo.gl/N1pDSp. Please feel free to apply there or send me your cv directly - dmitry.koval at company name com

Totally, buses never seem to stop at traffic lights.

Looking forward!

Any plans to add public transport support?

Flipboard and Prismatic are more visually appealing and intelligent options correspondingly. For mobile news readers category.

It always amazes me how people keep recommending CLRS as an interview preparation material. If you haven't studied Algorithms before it will take months to go through material in this book, leave alone making exercises. It is a serious textbook with strong mathematical rigor. If you have studied algos, granted, you would know what book to pick as a refresher and chances are - CLRS would be on your table. I think Sedgewick's book would be a more practical choice as a first book to learn some data structures/algorithms.

If you happen to be asked such questions it might be a good indication that the type they are looking for is code monkey. Or they are just not good enough to assess engineering skills. In both cases it's a red flag.

It seems you like you still hire based on puzzle as a first filter (based on on your careers page). Just wondering, what is the ratio of people who apply with only a resume vs those who solve the puzzle (I understand you probably can't reveal). And, more importantly, are those on the puzzles side show to be more successful on the interview?

In addition to the map tiles, OSM has an unbeatable advantage over any proprietery solution - you can take advantage of raw data and do whatever you can think of. For example, you can write your own route finding solution, tweak it whatever you like and potentially find new unexplored ways to existing problems (which I did for social based routing project in Eastern Europe). Needles to say - none of that is possible with the proprietary providers like Google.

This idea looks unreal for anyone thinking at least a tiny bit of long-term. It is not possible to provide all the infrastructure people need such as hospitals, schools, everything else and I seriously doubt anyone would want to put their families into such a circumstances when any of your entry to the US might be rejected.

Artima developer, http://www.artima.com/index.jsp - Java, Scala mostly

Have anyone done any performance comparison with the openjdk yet? GC related tests would be of high interest :)