jre

I like this idea very much. Shameless plug and I don't want to be the Rust fanboy, but I've played with something similar in Rust:

https://github.com/julienr/liveboard-rs

Basically it uses actix for the backend and yew (Vue-like rust frontend framework) for the frontend. This enables one to share types (and helper functions) between both, which is great:

https://github.com/julienr/liveboard-rs/blob/master/shared/s...

That being said, I think maturity-wise, Typescript is probably a better bet for this right now, so I'll definitely look at trpc for $dayjob.

Hey, I'm the lead engineer at Picterra. Would be happy to answer any questions.

What we are trying to do is indeed allow customers to build algorithms tailored to their use cases. We often joke that the reason our ML works is because we are overfitting to customer datasets/use cases. But I think that's somewhat true in the sense that we are not going for "worldwide trees counting" but more for "reliably count cars on these 50 parcels".

It's also interesting to note that a sizeable chunk of our revenue is coming from companies with drone data where it makes even more sense to be specific to customer data (resolution, time of day, geo, etc...).

Picterra | ML / Fullstack / Backend Engineers | Full-Time | REMOTE (EU only) or ONSITE (Lausanne, CH)

We are building a geospatial machine learning platform that allows users to detect any kind of object and train their own Deep Learning model in the cloud using an intuitive UI. That means we get to tackle interesting problems on a daily basis; from processing and visualizing large amounts of geospatial data to dynamically managing GPU workers, while exposing this to our users in a collaborative and intuitive web interface.

Our tech stack is Python/Django/Postgres on the backend and Vue on the front-end. We’re using PyTorch for our Deep Learning algorithms and deploy on Google Kubernetes Engine. We are working with geospatial data, so we are heavy users of GDAL, Mapbox, Leaflet, QGIS, and other GIS tools.

- Senior Full Stack Engineer: http://picterra.ch/senior-full-stack-engineer-2022/

- Junior Full Stack Engineer: http://picterra.ch/junior-full-stack-engineer-2022/

- Machine Learning Engineer: http://picterra.ch/machine-learning-engineer-2022/

- Backend / Devops Engineer: http://picterra.ch/backend-devops-engineer-2022/

To apply, send your resume to careers+eng [at] picterra.ch

I wonder if some of the classes are just easy to detect objects that they're using to assess the accuracy of the user.

I would guess they have a system such that after a user has passed N captchas successfully, they trust its a human and start displaying them (a portion of) unlabelled captchas that will always succeed and that's when novel labelling happens.

Or something along those lines. And then you can get creative displaying same captcha to multiple users, etc...

Picterra | Software Engineer, Backend & DevOps | Lausanne, Switzerland | Fulltime | Onsite | picterra.ch

We are a top tier VC-backed up start-up based in Lausanne, Switzerland. We are building the Google Search for the physical world. Our ambition is to give everyone–not just data scientists and developers–the opportunity to analyze and draw real-time insights from satellite and aerial imagery, all in just a few clicks.

We are looking for a Software Engineer with experience (~5 years) and interest in Backend/DevOps. You will join our small (currently 5) development team to help us scale our machine learning platform. Our scale challenges are mostly around data and efficient GPU utilization and not so much about number of concurrent users.

We have a Vue.js/Python/Django/DRF/Postgres/Docker/Kubernetes stack.

https://picterra.ch/about/

Picterra | Full stack Engineer | Lausanne or Berlin | Onsite | Full-Time | https://picterra.ch/

Looking for a Full-stack Software Engineer to join the Picterra team ! We build a web platform which allow our users to train custom object detectors for satellite/drone/aerial imagery. Our core idea is to use machine learning to make earth observation easier & more accessible.

We are still a small company (6), so now is the good time to join if you want to have an impact and be involved in product and engineering decisions.

We are working on quite cool stuff, with a highly interactive geospatial frontend (vue + leaflet + webgl). Our backend (python + django) ingests geospatial data and runs machine learning jobs on it and we deploy all of this to AWS with some scaling challenges ahead of us.

Apply through our website : https://picterra.ch/about/ or send me a mail at julien.rebetez [at] picterra.ch

I work at Picterra[1], a swiss company doing machine learning on drone & satellite imagery.

From discussion with a number of drone service providers, it seems to me the restrictiveness really depends on the context.

For things involving flying a drone over a crowded area, it's definitely hard to get authorization and we've seen projects being abandoned because of that. So yeah, drone delivery in cities might take some time before it is allowed.

On the other hand, most of the commercial applications I've seen are around using drones for mapping. So basically taking pictures and then processing them (create orthophotos, build a 3D model, detect objects, etc...). This usually doesn't involve flying over crowd and for a number of use cases (farms, construction site, security) the fly would be over a private property. A lot of people are able to do this without any legal restrictions.

From what I know of the history of the swiss Drone Valley, the two biggest success are sensefly [2] and Pix4D [3] and those are in the mapping business. Two of the booming startups would be wingtra [4] and flyability [5]. Wingtra seems to be mapping-oriented as well and Flyability develop drones that can fly inside. So a lot of the swiss drone valley is really into drones that can take pictures for mapping & inspection and I think this segment will continue to see massive growth in the future.

[1] https://www.picterra.ch/ [2] https://www.sensefly.com/ [3] https://pix4d.com [4] https://wingtra.com/ [5] https://www.flyability.com/

I'm not sure exactly what Netflix means by "if the job breaks", but I've used jupyter notebook in production to do ML before.

If you consider the notebook as a way to augment your logs with plots, it might make a bit more sense.

Running a jupyter notebook is a nice way to generate a HTML report for a job. For a typical ML pipeline, you first plot some stats about the input data, then train some model, plot some training loss, a confusion matrix, some example of predictions, etc...

If some job gives a strange result (maybe that's what they mean by break), having the notebook rendered as an HTML page with all the plot is a very effective way to do a first round of diagnostics. You can also start the notebook with the same parameters and 'run' through your report, which is a nice way to do interactive debugging.

Also in this case, the notebook itself was quite small in terms of lines of code. All the functions were implemented in modules, so it's really like the notebook is your 20 lines 'main' function. So you need some discipline among your team.

Those are all very good points I didn't think about, thanks for these.

I'll fix the two bugs with verify_block and the possibility for a miner to inject invalid a output > input transaction.

I'll add a note for the 3 others.

No good reason for RSA vs ECDSA. I was just more familiar with RSA, but apparently pycryptodome supports ECDSA as well, so I guess the change should be minimal.

Thanks for the precision regarding the hash.

Regarding the error, they are logged when a verify_block/transaction returns False, just to be a bit more explicit about what failed. In a real implementation, I guess you would throw exceptions instead (or use some Result pattern), but I tried and it cluttered the code quite a bit, so I went back to logging.

Thanks.

You're right about Merkle tree. This is a whole section of the bitcoin paper and it's pretty important. But as far as I understand, it's "only" an optimization to save disk space, so it doesn't change the underlying logic.

OP here.

Swapping MD5 for SHA256 is very easy. I'll actually do it - see my other answer above for why MD5.

For the other differences to bitcoin and from the top of my head :

- In my implementation, wallet addresses are the public key of the owner. Bitcoin addresses are slightly more complicated [1] and a wallet can (and should) generate a new address for each transaction.

- Bitcoin uses ECDSA instead of RSA

- Bitcoin transactions use a (simpler than ethereum but still) scripting language [2].

- The whole communication part was left out : you need a way to broadcast blocks. I haven't looked into that

- Bitcoin uses a Merkle tree to store transactions (and prune spent ones).

I think the scripting and communication would be the two biggest tasks. But it would also require unit testing and obviously wouldn't fit in a single notebook.

[1] https://en.bitcoin.it/wiki/Technical_background_of_version_1...

[2] https://en.bitcoin.it/wiki/Script

Thanks ! The awesome-blockchains is a great resource, thanks for sharing.

Thanks ! I've completely left out communication from this because it wouldn't fit in the notebook and I haven't researched it. Would also appreciate if anybody has good resources on it.

OP here.

erikb is spot on in the sibling comment. This hasn't been expert-reviewed, hasn't been audited so I'm pretty confident there is a bug somewhere that I don't know about.

It's educational in the sense that I tried as best a I could to implement the various algorithmic parts (mining, validating blocks & transactions, etc...).

I originally used MD5 because I thought I would do more exploration regarding difficulty and MD5 is faster to compute than SHA. In the end, I didn't do that exploration, so I could easily replace MD5 with SHA. I'll update the notebook to use SHA, but I'm still not gonna remove the warning :)

I'll also try to point out more explicitly which parts I think are not secure.

You forgot to mention that nodes always consider the longest block chain to be the consensus. A 51% (majority) attack is when one miner is able to produce more blocks than the rest of the network, therefore controlling consensus.

Such a miner could do a double-spend transaction by first spending on the short chain and then reverting his transaction on the longest chain. See https://en.bitcoin.it/wiki/Majority_attack

I don't know about the others, but the two visions dataset they compare to (MNIST and the face recognition one) are small datasets and the CNN they compare to doesn't seem very state of the art.

It also seems each layer of random forest just concatenates a class distribution to the original feature vector. So this doesn't seem to get the same "hierarchy of features" benefit that you get in large-scale CNN and DNN.

I built a small plugin to do that, it's called vim cell mode and provide "send selection to ipython" and "send block to ipython". I find the block mode very useful when developing data analysis scripts.

https://github.com/julienr/vim-cellmode

What company are you working at ? Looks like you do some interesting geospatial/map rendering stuff. Are you hiring ? :-)

Location: Lausanne, Switzerland

Remote: Yes

Willing to relocate: No

Technologies:

• Scientific Python : numpy, scipy, theano, tensorflow, keras (deep learning), vispy, mayavi, osgeo, gdal

• General Python : django, google appengine, flask, PyQT, PyOpenGL, cython, boost.python

• C/C++: OpenCV, Point Cloud Library (PCL), Eigen, OpenGL, Ceres

• Android : Sensors API (GPS, Accelerometer), Camera API, OpenCV4Android, NDK, OpenGL, Project Tango

• Big Data: Hadoop, HDFS, Spark

• Others : HTML/Javascript/CSS, d3js, PHP, Kinect SDK

Résumé/CV: http://fhtagn.net/files/resume.pdf

Email: julien.rebetez@gmail.com

Github: https://github.com/julienr

Website: http://fhtagn.net/

I am looking for Machine Learning and/or Computer Vision opportunities. In my day job, I currently work on a python machine learning platform for satellite images analysis. On the side, I have been hacking computer vision stuff using Google's Project Tango devkit.

I can't speak for caffe, but I got a new machine up and running with keras in about half an hour last week without having to compile anything :

- install the nvidia drivers, cuda and cudnn which all come as ubuntu packages.

- use anaconda for python

- conda install theano

- pip install tensorflow as per the website

- pip install keras

And voilà, I got my keras models running on my shinny new GTX 980.

I think the jupyter devs are aware of this and trying to find a fix. There is an enhancement proposal on how to get diff to work with notebooks : https://github.com/jupyter/enhancement-proposals/blob/master...

And I think that's the project that should implement this : https://github.com/jupyter/nbdime

Mesa has an offscreen rendering implementation that allows you to do OpenGL without an X server :

http://www.mesa3d.org/osmesa.html

Open a browser and navigate to your docker container's :8888 port. That would be localhost:9990 if you used the docker commandline given in the repo.

I stand corrected :) It seems like ILSVRC only has 2 datasets though, I wonder why they don't use a Kaggle-like approach ? My guess would be because they want people to be able to know their test score in order to put them in their papers.

They have a separate test set. But by submitting a lot of entries with slightly different parameters, you can optimize your parameters based on the test score.

Some (all ? ) Kaggle competition also have a daily submission limit to avoid this kind of cheating.

Not really. You can use widgets and redraw when the user changes something, but this gets clumsy quickly.

Bokeh allows for interactive visualization though. Vispy[1] does to, by using webgl (but this require a running ipython kernel - so the plot will be static when exported to HTML).

This[2] is an interesting blog post about the future of python plotting/visualization.

[1] http://vispy.org/

[2] http://www.almarklein.org/future_vis.html

I ran into the same issue. After a restore from my time machine backup, I discovered that some files where missing (with no apparent pattern). Fortunately, I had a non time machine backup, but pheew that was a close one...