HN user

mdda

1,343 karma

email me : {your.name} at mdda.net my blog : blog.mdda.net (AI and OSS)

Co-organiser of : https://www.meetup.com/Machine-Learning-Singapore/

Posts6
Comments618
View on HN

You've got to be careful with PDFs : We can't see how they are rendered internally for the LLM, so there may be differences in how it's treating the margin/gutters/bleeds that we should account for (and cannot).

"Make this better in a loop" is less powerful than using evolution on a population. While it may seem like evolution is just single steps in a loop, something qualitatively different occurs due to the population dynamics - since you get the opportunity for multiple restarts / interpolation (according to an LLM) between examples / and 'novelty' not being instantly rejected.

DeepSeek-R1 2 years ago

I think the "Aha" is that the RL caused it to use an anthropomorphic tone.

One difference from the initial step is that the second time around includes the initial step and the aha comment in the context : It is, after all, just doing LLM token-wise prediction.

OTOH, the RL process means that it has potentially learned the impact of statements that it makes on the success of future generation. This self-direction makes it go somewhat beyond vanilla-LLM pattern mimicry IMHO.

PyTorch 2.0 4 years ago

Google Colab gives you $free GPU (usually a 16Gb T4) preloaded with frameworks, ready to run. Later, you might be tempted by the Pro(+) version, but there's plenty of scope to move up the learning curve before spending any money.

To just play with something : https://huggingface.co/spaces/nateraw/yolov6 (There's an images tab, and some samples below).

If you go to the associated code, you'll see that it needs a 'backbone', 'neck' etc. What is a backbone? Questions that arise directly from the code will lead you towards good blog articles, etc. https://huggingface.co/spaces/nateraw/yolov6/blob/main/yolov...

OTOH, you could go and have a look at (for instance) the Stanford vision courses for a more 'theoretical' approach. But the code itself is often solid guide to what's going on (the frameworks used for Deep Learning map well onto what's being discussed in blogs/lectures/papers).

"All models are licensed under CC-BY-NC 4.0" :

So, to clarify, does this mean that companies cannot use these models in the course of business, or is it more about selling the translation results directly?

How DALL-E 2 Works 4 years ago

Could be... Except their page (should you choose to believe it, of course) specifically addresses the advantages:

"""

"Advantages over Traditional GANs" : Thus, we observe that our model exhibits _better training stability_ and mode coverage.

"Why is Sampling from Denoising Diffusion Models so Slow?" : After training, we generate novel instances by sampling from noise and iteratively denoising it _in a few steps_ using our denoising diffusion GAN generator.

"""

Failing at a business in the UK: "Told you so! And now everybody knows you're a failure."

Failing at a business in the US: "Every success has a few failures on the journey : If you have another go, it'll prove that you're a fighter!"

Succeeding at a business in the UK: "Who did you screw over to make that money?"

Succeeding at a business in the US: "Awesome! Let me pitch you my idea..."

The search term you're looking for is "Keyword Spotting" (or "Wake Word Detection") - and that's what's implemented locally for ~embedded devices that sit and wait for something relevant to come along so that they know when to start sending data up to the mothership (or even turn on additional higher-power cores locally).

Here's an example repo that might be interesting (from initial impressions, though there are many more out there) : https://github.com/vineeths96/Spoken-Keyword-Spotting

Look under "FP16 16-bit (Half Precision) Floating Point Calculations" on https://www.microway.com/knowledge-center-articles/compariso...

These raw numbers don't tell the whole story, of course. But IMHO, the convenience of a local 2080Ti outweighs the speed benefits of an _somewhat flaky_ V100 via Colab for day-to-day use (unless memory size is an issue, which you can't really get around).

OTOH, for just trying out stuff / one-offs, Colab is perfect - and bonus points if you score a V100.

Transputer 7 years ago

I was a summer intern at the UK company (Perihelion) doing the Atari-based Transputer machines. The word there was that because Atari had invested in the UK company (?) the Atari machine was essential to include as the front-end, even though it didn't really fit the UK designers' idea of what a good front-end machine would be... (nothing against the Atari design, just that its quirks/shortcuts didn't match up with the Transputer cluster backend 'vision').

The venue should be available for free if your meetup is attractive to people that could be potential employees of the hosts. And why do you feel the need to give people free food and drinks? Reframe the issue : Everyone who attends should be motivated to come to the event for the content, rather than free food on their way home.

~No. Suppose that 5 of 50 channels in a particular network layer made up the 'lottery ticket' for that layer. The number of 'combos' of 5 channels that were trained at the same time is 50C5 (i.e. ~2million [0]).

Whereas training 5C5 (=1) 10 times only gives you 10 chances to get the right 5 together.

At least, that's one way to think about it.

[0]: : https://www.mathway.com/popular-problems/Finite%20Math/60182...

Look at it from an interview perspective. If I ask "are you interested in exploring ML", and you're enthusiastic, my next questions are : What have you done? Have you taken any courses? GitHub? Blog Posts?

If the answer is that you're waiting for a special sign that it's worth doing before making an effort, then that really tells me that your enthusiasm for doing ML is not reality-based. Doing the ML thing is a pretty different mindset from other software jobs.