theikkila

Does WarpStream guarantee correct order inside partition only for acknowledged messages or also among the acknowledged messages (in different batches)? If so how do you keep clocks synchronized between the agents?

Related to 1. If I understood corrently the agent generates single object per each flushing interval containing all data accross all topics it has received. Does this mean that when reading the consumer needs to read multiple partition data simultaneously to access just single partition? How about scaling consumers horizontally how does WarpStream Agent handle horizontal partitioning of the stream from consuming side?

I have somehow feeling that this does not answer the original question. Could you give some examples of impossible or not easy tasks this achieves and other solutions don’t?

wasn't that 4k requests per minute?

I'm little bit troubled by the model created for classifying the vein patterns. With only corpus of size 40 and using two classes without augmentation will most likely end up overfitting the model. I would say the model is currently learning to classify left or right hand but doesn't really care about the veins much.

Have you tried the performance with some other user?

I would also probably use data augmentation and ie. flip and rotate images, vary contrast etc. That might prevent some amount of overfitting.

With these kinds of problems usually classifying models are not very well suited. Basically with neural networks you are causing the manifold to partition whole output space and so you can expect that there is practically unlimited amount of different patterns that look equal to your hand (the class you have trained to be 'your hand')

For better model you need more data, it can be labeled of course but there is also unsupervised options you could consider such as autoencoders. With facial recognition the siamese networks and triplet loss based networks are pretty popular and you could maybe take a look into them.

Not at all, deep nets are difficult to train and they need lot's of processing before they learn same kind of classifying features than ie. SVM has. So yes you can simulate SVM with deep networks but usually it's not very good solution.

SVM can be also used as part of the neural network such as in classifying layer

Hmm ok, yeah apparently you can leverage their API's and teach new labels for your own data.

https://azure.microsoft.com/en-us/services/cognitive-service...

That is great and will definetly help with problems where your task isn't just to recognize cats and dogs the only downside is that you are giving your data away and it will also help your competitors.

That sounds interesting, it's funny if you really can use their own models as base and do that. For the platform sake, Google offers also SaaS where you can train and evaluate your own models but then the base model is something you have to provide yourself

EDIT: I tried to google that up but couldn't find anything. Could you provide a link for that

I wanted to try out different types of final layers (SVM, Logistic Reg.) so the direct example didn't suit very well for that kind of testing out. For starters that good though.

Hello author of the article here!

I will probably publish in some time (when I'm not so busy doing other projects) some code samples too but until that I can give you tips.

You should start inspecting how the pre trained models (Inception v3/v4) works, what kind of layers they have and then decide what layers you want to use and what not. In case of Tensorflow, the tensorboard is very good tool for inspecting the model inner layers.

If you want to get started even more easier you should probably take a look into Tensorflow Slim models (https://github.com/tensorflow/models/tree/master/slim)

There is quite beginner friendly instructions for simple fine tuning of the models and it should take you pretty far.

I find this interesting to use even with common web applications as a design pattern. It's usually easy to just dispatch events but parsing them is always complicated and if you need to do that in all clients youre essentially decoupling the logic. If you can provide already parsed state you are providing the view into that data.

Crashed my chrome :D