karpathy

"""
The most atomic way to train and run inference for a GPT in pure, dependency-free Python.
This file is the complete algorithm.
Everything else is just efficiency.

@karpathy
"""

import os       # os.path.exists
import math     # math.log, math.exp

willccbb

# train_grpo.py
#
# See https://github.com/willccbb/verifiers for ongoing developments
#
"""
citation:

@misc{brown2025grpodemo,
  title={Granular Format Rewards for Eliciting Mathematical Reasoning Capabilities in Small Language Models},
  author={Brown, William},

beyonddream

Overview

Over a period of time, I started collecting technical papers that I would, given time and energy, like to implement an open source version of it. I use this to collect some such papers with the status of implementation for each. These papers have only one constraint and that is they should be simplistic enough to be able to implement within < 5000 LOC. I have also captured whether a paper has been implemented or not and if yes then in which language.

Papers

Papers	Implemented ?	Language	Repo
Automatic Keyword Extraction from Individual Documents by Rose et.al.	Yes	Java	https://github.com/beyonddream/JRAKE

o11c

Note: this was originally several Reddit posts, chained and linked. But now that Reddit is dying I've finally moved them out. Sorry about the mess.

---

URL: https://www.reddit.com/r/ProgrammingLanguages/comments/up206c/stack_machines_for_compilers/i8ikupw/ Summary: stack-based vs register-based in general.

There are a wide variety of machines that can be described as "stack-based" or "register-based", but not all of them are practical. And there are a lot of other decisions that affect that practicality (do variables have names or only address/indexes? fixed-width or variable-width instructions? are you interpreting the bytecode (and if so, are you using machine stack frames?) or turning it into machine code? how many registers are there, and how many are special? how do you represent multiple types of variable? how many scopes are there(various kinds of global, local, member, ...)? how much effort/complexity can you afford to put into your machine? etc.)

• a pure stack VM can only access the top elemen

yoavg

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of chatGPT, maybe played with it a little, and was imressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.

Intro

Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labour costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We

veekaybee

ChatGPT Resources

Context

ChatGPT appeared like an explosion on all my social media timelines in early December 2022. While I keep up with machine learning as an industry, I wasn't focused so much on this particular corner, and all the screenshots seemed like they came out of nowhere. What was this model? How did the chat prompting work? What was the context of OpenAI doing this work and collecting my prompts for training data?

I decided to do a quick investigation. Here's all the information I've found so far. I'm aggregating and synthesizing it as I go, so it's currently changing pretty frequently.

Model Architecture

CrockAgile

Floating Point Arcade

![Twitter Follow][twitter]

A Shady Coin Toss

A shady game master approaches...

Hey there! You look like you would enjoy a good game of chance.

ssrihari

An opinionated list of excellent Clojure learning materials

These resources (articles, books, and videos) are useful when you're starting to learn the language, or when you're learning a specific part of the language. This an opinionated list, no doubt. I've compiled this list from writing and teaching Clojure over the last 10 years.

• 🔴 Mandatory (for both beginners and intermediates)
• 🟩 For beginners
• 🟨 For intermediates

Table of contents

1. Getting into the language

emidoots

Because cross-compiling binaries for Windows is easier than building natively

I want Microsoft to do better, want Windows to be a decent development platform-and yet, I constantly see Microsoft playing the open source game: advertising how open-source and developer friendly they are - only to crush developers under the heel of the corporate behemoth's boot.

The people who work at Microsoft are amazing, kind, talented individuals. This is aimed at the company's leadership, who I feel has on many occassions crushed myself and other developers under. It's a plea for help.

The source of truth for the 'open source' C#, C++, Rust, and other Windows SDKs is proprietary

You probably haven't heard of it before, but if you've ever used win32 API bindings in C#, C++, Rust, or other languages, odds are they were generated from a repository called microsoft/win32metadata.

typesanitizer

Here is a list of resources that I have read either fully, or at least to an extent with which I am comfortable with endorsing them.

I've used these resources in different ways:

• Direct application: Some bits and pieces of advice can be applied very directly.
• Reflection: For organizational things, sometimes I would try to write down how I felt my work environment mirrored, and how it differed from the situation described in some work. Writing things down is a useful forcing function to think clearly.
• 1:1 discussions with my manager: We'd take the first 15-20 minutes of