danieljh

ruumi | Berlin, Remote (Europe timezones) | Full-time | https://www.ruumi.io

At ruumi we are building grassland and grazing management tools to give farmers actionable insights on how to manage their land sustainably and reduce their carbon emissions.

We are hiring for various roles

1. Engineers for web apps; React, TypeScript, Express, Postgres

2. Engineers for iOS and Android; Swift, SwiftUI, Jetpack, Jetpack Compose

3. Product owner for our apps and tools for farmers and carbon emissions

To apply see https://robofarm.jobs.personio.com or mail us at hello@ruumi.io and include your favorite animal emoji in the subject.

Learn more

- https://www.youtube.com/watch?v=fSEtiixgRJI

- https://www.imdb.com/title/tt8618654/

- https://eng.ruumi.io/post/seeing-through-clouds.html

- https://eng.ruumi.io/post/planet-scale-radar.html

The Future is Tiny Graphs! [1]

In the style of "Scalability! At What COST?" I've started a side-project bringing succinct data structures (rank/select [2], elias-fano coding) and modern instruction sets (popcount, pdep) to graphs.

For some reason the article doesn't mention the field of graph compression and I think it shows: for example most if not all popular open source routing engines don't do anything regarding graph compression. It's time to change that! Reach out to me (preferably on the Github project) if you want try some ideas :)

[1] https://tinygraph.org [2] http://www.cs.cmu.edu/~dga/papers/zhou-sea2013.pdf

ruumi | Berlin, Remote | Full-time | iOS, Swift, SwiftUI | https://www.ruumi.io

At ruumi we help farmers with rotational grazing: a managed grazing technique that regenerates soil, increases biodiversity, minimizes agro-chemicals, sequesters carbon, and improves profitability at the same time.

We are bringing multi-spectral and radar satellite imagery, geo-spatial tech, and machine learning to regenerative agriculture. We tell farmers actionable insights for example: when and where to move their animals to maximize soil regeneration and carbon sequestration.

We are looking for someone to work with us on our applications, with a strong focus on iOS.

Check out https://www.notion.so/ruumi/ruumi-jobs-86745c0df9834395b27f0... https://eng.ruumi.io and send me your favorite animal emoji!

ruumi | Berlin, Remote | Full-time | Frontend, Mobile | https://www.notion.so/ruumi-jobs-86745c0df9834395b27f018fec8...

At ruumi we help farmers with rotational grazing: a managed grazing technique that regenerates soil, increases biodiversity, minimizes agro-chemicals, and improves profitability at the same time.

We are bringing multi-spectral and radar satellite imagery, geo-spatial tech, and machine learning to regenerative agriculture. We tell farmers actionable insights for example: when and where to move their animals to maximize soil regeneration.

We are looking for someone to work with us on web and mobile applications. For the frontend we are using Typescript, React and Redux; for the backend we are using Typescript and Express.js.

Check it out! https://www.notion.so/ruumi-jobs-86745c0df9834395b27f018fec8... and https://eng.ruumi.io

ruumi | Berlin, Remote | Full-time | Frontend & Mobile | https://www.notion.so/ruumi-jobs-86745c0df9834395b27f018fec8...

At ruumi we help farmers with rotational grazing: a managed grazing technique that regenerates soil, increases biodiversity, minimizes agro-chemicals, and improves profitability at the same time.

We are bringing multi-spectral and radar satellite imagery, geo-spatial tech, and machine learning to regenerative agriculture. We tell farmers actionable insights for example: when and where to move their animals to maximize soil regeneration.

We are looking for someone to work with us on web and mobile applications. For the frontend we are using Typescript, React and Redux; for the backend we are using Typescript and Express.js.

Check it out! https://www.notion.so/ruumi-jobs-86745c0df9834395b27f018fec8... and https://eng.ruumi.io

ruumi | Berlin, Remote | Full-time | Frontend & Mobile Engineer | http://jobs.ruumi.io

We are looking for someone to work with us on web and mobile applications. For the frontend we are using Typescript, React and Redux; for the backend we are using Typescript and Express.js.

At ruumi we help farmers with rotational grazing: a managed grazing technique that regenerates soil, increases biodiversity, minimizes agro-chemicals, and improves profitability at the same time.

We are bringing multi-spectral and radar satellite imagery, geo-spatial tech, and machine learning to regenerative agriculture. We tell farmers actionable insights for example when and where to move their animals to maximize soil regeneration.

Check it out at http://jobs.ruumi.io

ruumi | Berlin, Remote | Full-time | Frontend & Mobile Engineer | http://jobs.ruumi.io

We are looking for someone to work with us on web and mobile applications. For the frontend we are using Typescript, React and Redux; for the backend we are using Typescript and Express.js.

At ruumi we help farmers with rotational grazing: a managed grazing technique that regenerates soil, increases biodiversity, minimizes agro-chemicals, and improves profitability at the same time.

We are bringing multi-spectral and radar satellite imagery, geo-spatial tech, and machine learning to regenerative agriculture. We tell farmers actionable insights for example when and where to move their animals to maximize soil regeneration.

Check it out at http://jobs.ruumi.io

Daimler Autonomous Services | ONSITE, Full-Time | Berlin, Germany | VISA Sponsorship

We are building AV-enabled routing and mapping capabilities to help ride-hailers and car manufacturers around the world shape the future of autonomous mobility. Together with Daimler's autonomous and vehicle teams our goal is not simply to get our customers from A to B but to get them there with the quality and safety they have come to expect from our brand.

We are hiring for:

* Backend - Node.js, Swagger/OpenAPI, k8s; to work on our services and the underlying infrastructure

* Routing & Maps - Python, C++, Rust; to work on graphs, shortest path algorithms, machine maps / lane-level maps

* Machine Learning - PyTorch, TensorFlow; to work on feature extraction, graph embeddings, traffic models, and demand prediction

Some example projects our engineers have worked on before:

* https://github.com/Project-OSRM/osrm-backend

* https://github.com/mapbox/robosat

Bonus points if you have worked professionally with geospatial data and software before and/or have contributed to OpenStreetMap, be it code or map edits.

Write me a note at daniel(dot)hofmann(at)daimler(dot)com telling me about what projects you've worked on that fit what we're looking for. Include a robot emoji in the subject to confirm you are not a robot ;) You can also find us and talk to us engineers at local meetups such as Geo Berlin.

We are running it internally on our aerial imagery from the Mapbox Maps API. The zoom level even there depends on the feature you want to extract, for example z18 seems to work well for parking lots.

There is not a single feature this model is most suited for: you can add arbitrary features (e.g. tennis courts, swimming pools) in pre-processing and train your model. Then the imagery quality depends on your feature, for example it will be hard to impossible to spot swimming pools in Landsat imagery.

We haven't released pre-trained models yet. Mostly for two reasons: 1/ The PyTorch checkpoints depend on the specific Python model class. Even if you refactor only e.g. a MaxPool layer into a direct functional.max_pool function call, loading old checkpoints will no longer work. We have an ONNX model exporter now (rs export) which allows for self-contained and portable protobuf model and weight files. This workflow needs some more time and careful evaluation, though. 2/ The models for Tanzania I was working on in my spare time I can open up for sure. If there is community interest maybe we can come up with a publicly available model catalogue hosting ONNX models and metadata where folks can easily upload and download models. For our internal models and the data we extract we are thinking through a broader strategy since a lot of time and resources are going into creating and cleaning datasets, doing hard-negative mining, running multiple training iterations and so on. They're also bound to the Mapbox aerial imagery on specific zoom levels.

The model architecture is kept simple on purpose. It used to be an encoder-decoder U-Net'ish architecture which we trained from scratch. Recently (https://github.com/mapbox/robosat/pull/46) I switched out the encoder to a pre-trained ResNet, as proposed by Alexander Buslaev. It's a mix of the papers listed in the docstring at the top with a focus on simplicity and maintainability:

https://github.com/mapbox/robosat/blob/1e687552fe9b254a14d55...

Internally we were also exploring a multi-class PSPNet but decided not to move forward with it right now: the RoboSat model is currently a binary model (feature vs. background) which makes a few things easier in practice, such as efficiently storing results which is needed when scaling it up e.g. to all of North America.

I haven no experience with TensorFlow.js. That said, using the RoboSat ONNX model exporter (rs export) you should be able to go from a trained PyTorch model to a portable ONNX protobuf, then from there to a TensorFlow model, and eventually to TensorFlow.js. At least that's how I would approach it. Keep me posted if you look into it and get it working, interesting use-case for sure.

Daniel from Mapbox here. Happy to answer questions or talk through design decisions. Interested to hear your feedback.

We are mostly focusing on making the process accessible to a broader audience in the geo space, building a solid production-ready end-to-end project.

There are more resources and a step-by-step guide for running on openly available drone imagery in Tanzania:

https://www.openstreetmap.org/user/daniel-j-h/diary/44145 https://www.openstreetmap.org/user/daniel-j-h/diary/44321

Sentinel 2 data could be interesting for you. With its high refresh rate it can be used for change detection. The multi-band data is especially useful for adding more input channels to the model e.g. for water detection based on NDWI: https://apps.sentinel-hub.com/eo-browser/#lat=52.51747&lng=1...

I can also recommend http://openaerialmap.org for playing around with openly available drone imagery. I did this last week as a side-project on my evenings for drone imagery provided by http://www.zanzibarmapping.com. Here is a step-by-step guide for giving it a try on the Tanzania drone imagery: https://www.openstreetmap.org/user/daniel-j-h/diary/44321

Here is how I build architecture optimized TensorFlow 1.4 binaries for specific AWS instance types:

https://gist.github.com/daniel-j-h/b410dd21d4949a54c83d0c386...

The build script needs slight adaptions between major TensorFlow releases; otherwise works great for me so far.

During boot time you can then figure out the instance architecture you are and fetch the appropriate wheel. Hope it helps.

Check out metric trees such as the BK-Tree, VP-Tree or GNATs. They allow for fast nearest neighbor searches, for example querying all items in your database with Hamming Distance h < 2.

I stumbled upon metric trees for nearest neighbor queries in metric spaces some months ago and wrote down notes here:

http://daniel-j-h.github.io/post/nearest-neighbors-in-metric...

Hope that helps.

Absolutely. The following two new checks are impressive and should be on by default.

New misc-move-forwarding-reference check Warns when std::move is applied to a forwarding reference instead of std::forward.

http://clang.llvm.org/extra/clang-tidy/checks/misc-move-forw...

New misc-use-after-move check Warns if an object is used after it has been moved, without an intervening reinitialization.

http://clang.llvm.org/extra/clang-tidy/checks/misc-use-after...

The use-after-move check has its limitations which make sense considering the "unspecified but valid state" moved from objects are in. From reading the design document the cases they handle should catch common ownership issues, though. Great step in the right direction for sure.

Implementation: https://github.com/llvm-mirror/clang-tools-extra/blob/master...

Related: https://doc.rust-lang.org/book/ownership.html

Here's an interesting anecdote for -ffast-math: the Kahan summation algorithm [1] can be used to reduce numerical errors in sums.

Now have a look at the codegen [2] without and with -ffast-math.

With -ffast-math the compiler rewrites the Kahan summation algorithm to a simple sum, completely screwing the code's semantic (since we told the compiler to do so!). Keep this in mind.

1 https://en.wikipedia.org/wiki/Kahan_summation_algorithm 2 https://godbolt.org/g/uWZgW2

where's all the actual OS research? Where's the service-oriented OS built on capabilities with a typed object store for a filesystem?

Once you realize there's not much a difference between Microkernels are the recent microservices architectures, they become interesting again. Then you might want to look in recent advances such as Unikernels or Anykernels.

Here's a good start summarizing lessons in Microkernels learned in the last decades: http://dl.acm.org/citation.cfm?id=2522720

The sliding shortest path paper is really interesting.

I'm thinking about what this scenario means for road networks and if it makes sense there at all (not considering the algorithm's assumptions). I guess it would answer the question of how bad traffic jams have to be on certain roads in the shortest path in order to force the route over a specific road.

On a sidenote: in the caption for figure 2, it has to be "link ef is cut" instead of "link eg is cut" in b) if I understand it correctly.

From what I can see in the scientific community it seems like TBB does not get the attention it deserves.

In my opinion this comes from two simple facts: 1/ it's easier to throw a OpenMP #pragma on your loop after detecting the bottleneck with a profiler and 2/ you need to understand some TBB concepts like partitioning/splitting to implement your own algorithms. That is, TBB much more integrates into the language (modern C++ with lambdas etc.).

I won't tell you much about TBB's concurrent containers (maps, queues, vectors, ...), parallel algorithms (sort, scan, reduce, ...), or memory allocators. Those are explained in detail in the documentation. What I want to tell you about is TBB's Pipeline and FlowGraph feature, because of how powerful they are and how often they are simply ignored.

TBB's pipeline lets you build a pipeline of parallel or serial stages. On a high level it really is function composition, with the benefit of TBB deciding on the level of parallelism.

Here is an example of using TBB's pipeline to receive from the network, deserialize the blob and merge the results concurrently and potentially in parallel:

https://github.com/daniel-j-h/DistributedSearch/blob/97224b1...

You can find a quick explanation in this blog post:

https://daniel-j-h.github.io/post/intuitive-monadic-bind-kle...

All you have to do is create your stages as functions: take input from stage n-1, process it and move ownership of the item over to stage n. Note: C++11's move semantics do not require you to pass raw pointers around or do the memory management yourself, as it is done in TBB's documentation!

As you can see, you only need the parallel_pipeline function (and make_filter<In, Out> for when the stages are not passed directly as lambdas):

https://www.threadingbuildingblocks.org/docs/help/reference/...

TBB's pipeline is really powerful for scenarios where a linear processing chain is needed, e.g. face detection with OpenCV where you have to 1/ grab a frame 2/ do some histogram corrections 3/ apply gaussian blur 4/ apply a face detection algorithm 5/ merge face's position into circular buffer 6/ do average over this buffer to estimate face position.

TBB's FlowGraph is for when your dependencies are more difficult to express as a simple pipeline. In the OpenCV example, maybe you need to buffer 5 frames for the face detection, maybe you need to join inputs from a camera and a video and react on when a face is found in a camera frame.

An example for expressing arbitrary dependencies is here:

https://www.threadingbuildingblocks.org/docs/help/reference/...

And an example for joining two nodes:

https://www.threadingbuildingblocks.org/docs/help/reference/...

This is the documentation's starting point for FlowGraph:

https://www.threadingbuildingblocks.org/docs/help/index.htm#...

There is also a book about TBB out there and it contains some additional examples. But as it is with most of the documentation, those examples are not written in modern C++ (C++11/C++14) and the book is a bit dated.

The reason I did not show code is that there is 1/ no notion of a monad type class in C++ 2/ no std::optional in the current C++14 standard and 3/ you have to work around side effects, making an actual implementation highly dependent on your environment and so on.

I'm playing around with what is now my second iteration of Monadic Bind and Kleisli Composition for boost::optional, which looks something like this right now:

    // tag dispatch work around for void returning functions (that make no sense in a purely functional world)
    namespace detail {
        template <typename A, typename Fn>
        auto monadicBindImpl(const boost::optional<A>& a, Fn f, std::false_type) -> decltype(f(*a)) {
            if (a) return f(*a); else return {};
        }
   
        template <typename A, typename Fn>
        auto monadicBindImpl(const boost::optional<A>& a, Fn f, std::true_type) -> void {
            if (a) f(*a);
        }
     }

    template <typename A>  
    auto pure(A&& a) { return boost::optional<A>(std::forward<A>(a)); }  

    template <typename A, typename Fn>  
    auto operator>>=(const boost::optional<A>& a, Fn f) -> decltype(f(*a)) {
        return detail::monadicBindImpl(a, f, typename std::is_same<decltype(f(*a)), void>{});
    }  

There are other articles on the web for Monads in C++, that probably do a better job at an actual implementation. Please see this as my personal playground code :)

For C++, unfortunately CMake is the best option there is it seems. For LaTeX there is latexmk. Clojure has Leiningen.

You see, it heavily depends on your preferred environment.

Implicit makefile rules can be quite helpful in quickly building a few binaries or small projects.

Take a look at this C++ example: for a file mybin this creates mybin.o and then links it, spitting out a mybin executable:

    env CXXFLAGS="-std=c++14 -Wall -Wextra -pedantic" LDLIBS="-lstdc++" make mybin

    make --print-data-base | egrep 'COMPILE.cc|LINK.cc'

For small projects I would create a config.mk (e.g. with CXXFLAGS, CXX, ...) and provide a small Makefile, something along the lines of:

    include config.mk

    all: mybin

    mybin: mybin.o

    watch:
        while ! inotifywait -e modify *.cc; do make; done

    clean:
        $(RM) *.o mybin

    .PHONY: all watch clean

Disclaimer: Makefiles seem to be wonderful build-systems (not dependency managers!) for small projects; but as soon as you find yourself searching for hacks to build into your little Makefile, ditch it and switch to something serious. That's at least my experience.

I'm trying Rust for the same reason why I'm investing weeks into learning Clojure. Because I'm convinced learning more approaches to high-level problems (like ownership, concurrency, typing, patterns) is one way to improve my overall skill set and thinking.

One thing I like about Rust is how you e.g. can specify a Sortable trait and then with T: Sortable require it for types. This will eventually come to C++ with Concepts (lite), for now there are ways to work around this, e.g. with:

    #define requires(...) typename enable_if<(__VA_ARGS__), int>::type = 0

    template <typename T, requires(is_integral<T>())>
    void fn(T);

Your second function still mixes ownership with resource management. Taking an unique_ptr<BigInteger> by value means, add_one takes the ownership of it. But it does not return anything, making it a useless function.

The though-process goes like this:

* Do you need read-only access to the argument, take a const T&

* Do you need to modify the argument, take a T (by value), letting the call site decide to either pass an rvalue (move it, no copy), or an lvalue (copy)

There simply is no need for unique_ptr<BigInteger>, as BigInteger already handles its resources internally (with move semantics). It's the same reason as to why an owning pointer to a vector is silly when a vector already handles ownership of its resources.

http://klmr.me/slides/modern-cpp/#9 is exactly about this.

Accessing a moved-from object is one of the examples I didn't criticize because it could be an issue and I hope for compilers warning about it. That doesn't make my other claims invalid, though.

I'm not missing the point of those examples. They showcase Rust vs. ridiculous C++.

See my point with making the add_one function not taking a unique_ptr. This is valid criticism and valid in more complex practical examples, too.

I'm bored of all the Rust vs. C++ comparisons, when the C++ code is something you simply don't write with modern and idiomatic C++11 and C++14. Not that you could not write it, but seeing this would not pass me reviewing the code.

The add_one example, as simple as it is, is a horrible example.

Why not embrace value semantics and write it like:

    int add_one(int x) { return x + 1; }

    auto x = std::make_unique<int>(5);

    int x{5};

    void add_one(std::unique_ptr<int> num) {
        * num += 1;
    }

    int add_one(int); // from above

    auto p = make_unique(5);
    auto six = add_one(*p); // <-

For more about modern practices see: http://klmr.me/slides/modern-cpp/

Also see Howard Hinnant's unique_ptr tutorial: http://howardhinnant.github.io/unique_ptr03.html

As someone looking more into x86_64 assembly, instruction encoding, syscalls and ELF files recently, not only the lack of good starting points but also the amount of work required to get into it is a pity.

I'm currently using [0] as a helping hand among other resources which is quite good; my maybe not-so-interesting results are at [1].

For example, to get a good overview of instruction encoding you have to 0/ read through and ditch horrible blog posts 1/ find the correct Intel manual 2/ search and read through thousands of PDF pages until you find something interesting 3/ understand the environment and facts that are either implicitly given or in the documents but not easy to find.

For the handful lines of actual code I wrote yesterday [1] I still have around 25 tabs open. Complexity and no end in sight.

Do you have any recommendations and hints as where to start with this in the year 2015?

[0] http://0xax.blogspot.se/p/assembly-x8664-programming-for-lin...

[1] https://github.com/daniel-j-h/x86-64-asm

If you want to learn by examples, check out the following repositories:

https://github.com/daniel-j-h/cpp14-snippets https://github.com/daniel-j-h/cpp11-snippets

(Disclaimer: the C++14 version is still missing proper references to the official C++14 standard)