DGoettlich

data is 100% public domain.

very interesting observation!

well put.

thanks. i think this just took on a weird dynamic. we never said we'd lock the model away. not sure how this impression seems to have emerged for some. that aside, it was an announcement of a release, not a release. the main purpose was gathering feedback on our methodology. standard procedure in our domain is to first gather criticism, incorporate it, then publish results. but i understand people just wanted to talk to it. fair enough!

i think we (whole section) are just talking past each other - we never said we'll lock it away. it was an announcement of a release, not a release. main purpose for us was getting feedback on the methodological aspects, as we clearly state. i understand you guys just wanted to talk to the thing though.

Thanks for the comment. Could you elaborate on what you find iffy about our approach? I'm sure we can improve!

valid point. its more of a stepping stone towards larger models. we're figuring out what the best way to do this is before scaling up.

we're on the same page.

exactly

Also one of our fears. What we've done so far is to drop docs where the datasource was doubtful about the date of publication, if there are multiple possible dates we take the latest to be conservative. During training, we validate that the model learns pre- but not post-cutoff facts. https://github.com/DGoettlich/history-llms/blob/main/ranke-4...

If you have other ideas or think thats not enough, I'd be curious to know! (history-llms@econ.uzh.ch)

what makes you think we trained on only a few gigabytes? https://github.com/DGoettlich/history-llms/blob/main/ranke-4...

fully understand you. we'd like to provide access but also guard against misrepresentations of our projects goals by pointing to e.g. racist generations. if you have thoughts on how we should do that, perhaps you could reach out at history-llms@econ.uzh.ch ? thanks in advance!

understand your frustration. i trust you also understand the models have some dark corners that someone could use to misrepresent the goals of our project. if you have ideas on how we could make the models more broadly accessible while avoiding that risk, please do reach out @ history-llms@econ.uzh.ch

thanks. we'll be more precise in the future. ultimately, we took whatever we could get our hands on, that includes newspapers, periodicals, books. its multilingual (including italian, french, spanish etc) though majority is english.

that is simply how we display the questions, its not what the model sees - we show the chat-template in the SFT section of the prerelease notes https://github.com/DGoettlich/history-llms/blob/main/ranke-4...

we were considering doing that but ultimately it struck us as too sensitive wrt the exact in context examples, their ordering etc.

We tell it that its a person (no gender) living in <cutoff>: we show the chat template in the prerelease notes https://github.com/DGoettlich/history-llms/blob/main/ranke-4...