tonicinhibition

tonicinhibition t1_jdn4v86 wrote

Reply to comment by Veggies-are-okay in [N] March 2023 - Recent Instruction/Chat-Based Models and their parents by michaelthwan_ai

There's a YouTuber named Letitia, with a little Miss Coffee Bean character, who covers new models at a decent level.

CodeEmporium does a great job at introducing aspects of the GPT/ChatGPT architecture with increasing depth. Some of the videos have code.

Andrej Karpathy walks you through building GPT in code

As for the lesser known models, I just read the abstracts and skim the papers. It's a lot of the same stuff with slight variations.

6

tonicinhibition t1_jb1ntqz wrote

Reply to comment by currentscurrents in To RL or Not to RL? [D] by vidul7498

I don't think the author of the post took a position on the original argument; rather they just presented ways to explore the latent space and make comparisons that are reasonable so that we might derive better distance metrics.

I see it as a potential way to probe for evidence of mode collapse.

1

tonicinhibition t1_jb1fgpe wrote

Reply to comment by tripple13 in To RL or Not to RL? [D] by vidul7498

> people who discount GANs due to their lack of a likelihood

I was going to ask you to expand on this a little, but instead found a post that describes it pretty well for anyone else who is curious:

Do GANS really model the true data distribution...

For further nuance on this topic, Machine Learning Street Talk discussed interpolation vs extrapolation with Yann LeCun regarding interpolation vs extrapolation, which Letitia Parcalabescu summarizes here.

1