index the codebase first. Then ask o1 in "codebase chat" the same questions you would like to ask the author/owner of the codebase.

Mostly useful when digging into unknown/new codebases and trying to understand them. Or asking about possible bugs :)

yes, I noticed that over Christmas break and ever since, it's just... a lot more boring here.
My first reason to open social media is to be entertained. Second reason to entertain. Maybe last third to learn something new.
Not much of any of these here. I know I know, be the change and all that.

It reeeeally depends what are loss1 and loss2, both regarding what’s standard and what’s wasted.
I honestly think you are confused, the three codes in three different posts of you mean three different things. I don’t mean it in a negative way, but clearing it up would take more time than I want :/

Yeah it seems either he had a mistake in the OP, or the subject of the discussion has drifted :)

Until we got good enough AI supported search, no, you can’t realistically expect them to find anything and everything from the past 30 years when the vocab and everything changes.

Well, no, two things:

1. In the OP indeed *both* formulations waste compute, so yeah :)

2. In 2nd post, you are not doing the same thing as in your OP! In your 2nd, you are doing good old micro batching which indeed the second way is the standard way.

So what you say keeps changing O.o

I would enjoy that meeting ;)

Yeah it’s silly to expect the new generation to know everything the old generation did, doing so shows complete lack of empathy.

If the two graphs are completely disjoint, then there is no point in this. If they have some commonality (like model) then this does the common part twice.

I’m somewhat confident both of these are sins lol second one wastes a ton of compute!

Not quite because this one is not stacked, so I give it better chance to scale:

That’s what the globe was for!

One of the physics of llm papers studied that and found you need a certain amour of repetitions of a factoid before it’s memorized. Repetition can be either multi epochs or just the same fact in another document. Number of needed repeats is also related to model size.

OK OK I’ll admit it, I’m feeding off your fomo! There can’t be enough fomo! Mmmm fomo!

Yeah they compress videos to shit here, and are considering making good quality videos a paying feature.

No talk but two posters, I’m just a middle author but will try to be there (locca and NoFilter). That being said my main occupation here will be meeting many of my new colleagues.

lol exactly. I said cdg whenever possible.

That being said, our flight is operated by AirCanada, including a layover in Canada, that is a lot worse.

Oops! Thanks

This afternoon flight? I’m taking that too