HNGlance

Top Best New Show

Code Walkthrough of Bert with PyTorch for a Multilabel Classification in NLP

https://engineering.wootric.com/when-bert-meets-pytorch

by prabhatjha • 7 years ago

13 2 7 years ago

EN

engineering.wootric.com

Related Stories

Loading related stories...

Comments

2

rsmith49• 7 years ago

For fine tuning BERT onto a specific domain, what amount of text data would you recommend to train on?

yashvijay• 7 years ago

Since the training is done on the tasks of masked word prediction and contiguous sentence prediction, I'd suggest about a million sentences (from the same domain), with an average token length of 7 per sentence. Longer sentences would definitely help, as BERT uses the transformer encoder architecture which has multi head attention. This would enable the model to do better contextual representation learning for the embedding layer.

Source preview

engineering.wootric.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms