HN user

elyase

170 karma

email: yaser.martinez at (the famous google mail) twitter: elyase

Posts11
Comments68
View on HN

This is similar to the tool call (fixed code & dynamic params) vs code generation (dynamic code & dynamic params) discussion: tools offer contrains and save tokens, code gives you flexibility. Some papers suggest that generating code is often superior and this will likely become even more true as language models improve

[1] https://huggingface.co/papers/2402.01030

[2] https://huggingface.co/papers/2401.00812

[3] https://huggingface.co/papers/2411.01747

I am working on a model that goes a step beyond and even makes the distinction between thinking and code execution unnecessary (it is all computation in the end), unfortunately no link to share yet

What you're missing is how to use the tools properly. With solid documentation, good project management practices, a well-organized code structure and tests, any junior engineer should be able to read up on your codebase, write linted code following your codebase style, verify it via tests and write you a report of what was done, challenges faced etc. State of the art coding agents will do that at superhuman speeds.

If you haven't set things up properly (important info lives only in people’s heads / meetings, tasks dont have clear acceptance criteria, ...) then you aren't ready for Junior Developers yet. You need to wait until your Coding Agents are at Senior level.

We have done extensive testing in the context of chatbot intent classification and in our particular problem nothing (including CNN, LSTM, fasttext plus LUIS, Watson and other proprietary classifiers) has been able to beat a simple linear model trained on char n-gram features.

fast.ai is great and I also want to love it but I find the coding style extremely unconventional and uninviting to the point that I only use the library when there is really no better alternative (ex for the lr finding features). Variable naming and wildcard imports would be my main complains. But thanks a lot Jeremy for the course and for posting the style guide.

We evaluated using ConceptNet Numberbatch but in the end went with fasttext because of the treatment of OOV words using sub word information. This is important for us because we work with Social Media where misspellings are very frequent and we have found this helps a lot. Are you also looking into these sort of enhancements? How do you usually deal with OOV words?

If you follow the right people in your fields of interest you will have an endless stream of interesting updated content. You can't personalize HN to have the exact sort of news you want but you can tune the people you follow in Twitter and get the best source of curated news there is for you. I frequently find stuff like new library releases/interesting blog posts from someone I follow well before they appear (if they ever do) in places like HN/Reddit subs I frequent.

Nootropics 10 years ago

A study assessing 6 months (n=112) and 12 months (n=96) of melatonin treatment of 2mg in a controlled release capsule, taken 1-2 hours prior to sleep, people aged 20-80 with primary insomnia failed to show any tolerance to the treatment. The authors noted a slight sensitization to the effects of melatonin at the 3-4 month period, which was attributed to better entrainment of the circadian rhythm. These results have been replicated in another study, lasting 6 months, with a sample of 791 people. No melatonin tolerance due to usage was observed. Another study, lasting 6 months with a sample size of 421 people also replicated these results.

Source: http://examine.com/supplements/Melatonin/#summary15-1

I think this could have been done with GNU parallel. One advantage I see with Spark is that is that it is easier to interact with Python, for example these two lines are all is needed to call the relevant Python function:

  urls = sc.parallelize(batched_data)
  labelled_images = urls.flatMap(apply_batch)
So if you already have a cluster with Spark installed (like Databrick does) then it takes less work to just call your Python code than setting up a GNU Parallel cluster and a writing a small wrapper script. Additionally a Python script would have to load/init the models on every call from Parallel. I agree that this is not a great demonstration of Spark main strengths.

Essentially spacy is better and faster at almost everything it supports but is not free for commercial use. I consider spacy a blessing for the Python and the NLP community in general. They have a great comparison with existing libraries at http://spacy.io/.

EDIT: From today on spacy is free for commercial use! (MIT license).

Just for comparison this is almost the same using a library [1]:

  import numpy as np
  X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1]])
  y = np.array([[0,0,1,1]]).T

  from keras.models import Sequential
  from keras.layers.core import Dense

  model = Sequential([Dense(3, 1, init='uniform', activation='sigmoid')])
  model.compile(loss='mean_absolute_error', optimizer='sgd')
  model.fit(X, y, nb_epoch=10000)
  model.predict(X)
[1] http://keras.io

Sorry if it sounded like that. What I meant is that the 2d matrix representation with shape (n_samples, n_features) actually goes beyond scikit-learn and python(ex: dataframes in R or Julia), it is the standard representation of data in Machine Learning so it is assumed that someone who wants to do Deep Learning should already be familiar with it. That is why I thought you should start with something simpler than Deep Learning to get used to these concepts. Scikit-learn is a good option because it has more tutorials/examples/videos and more beginner friendly documentation in general.

Yep, as he mentioned you need some familiarity with scikit learn or similar APIs, take a look at [1] for example. In essence X_train and Y_train are 2d arrays with shape (n_samples, n_features), Y_train is usually of shape (n_samples, 1) the same as the output. Normally both list of lists or numpy arrays are accepted, even a generator of samples as long as it is a 2D like structure. I would say that if this is not obvious to you maybe you should start with something more basic like linear models in scikit-learn before jumping to deep learning.

[1] http://scikit-learn.org/stable/tutorial/basic/tutorial.html#...