Skip to content

Instantly share code, notes, and snippets.

View natevw's full-sized avatar

Nathan Vander Wilt natevw

View GitHub Profile
@celoyd
celoyd / antisdpa.md
Last active January 15, 2026 21:56
The usual implementaiton of attention transformers (SDPA) is kind of bad, actually

Introduction

I was writing a note to a friend that mentioned my tedious opinions on “AI” discourse. It veered off into my usual argument that big “AI” companies are shaping the industry ecosystem to their own ends by setting up a situation where expensive-to-run models are overvalued. I think they’re doing this because they have a competitive advantage in that tier of the market, having bought (time on) a lot of GPUs. It’s like how a company that owns diamond mines will probably promote the idea that large, mined diamonds are important and valuable, and that there’s something off about running a sub-industrial mine or lab-growing diamonds. You can do this without lying at all, but I still dislike it. Large mined diamonds here are $O(n^2)$ models.

To support this argument, I started making my case against the necessity of the standard transformer model. I admit that the case is scattershot and circumstantial. It’s not that SDPA (the normal transformer architecture) is a fraud, or that there is something m

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@getify
getify / 1.js
Last active September 29, 2021 11:58
experiment: mimicking React's new "useState()" hook for stand-alone functions, including "custom hooks"
"use strict";
[foo,bar] = TNG(foo,bar);
// NOTE: intentionally not TNG(..) wrapping useBaz(), so that it's
// basically like a "custom hook" that can be called only from other
// TNG-wrapped functions
function foo(origX,origY) {
var [x,setX] = useState(origX);
var [y,setY] = useState(origY);
@felipeochoa
felipeochoa / quill-blots.org
Created January 13, 2018 17:13
Notes to self on blots in quill

Quill Blots

There are four options for creating a custom blot:

---------------+-------+

InlineBlock

---------------+-------+

Embed

---------------+-------+

Text
@tmcw
tmcw / jargon.md
Last active October 5, 2016 14:39
Startup English
This team is growing We're hiring lots of people
"Hunger" Enthusiasm
"Fire" Enthusiasm
"Spark" Enthusiasm
Commander's intent Goals
Onboarding Training
The best Pretty good
Burn Money-losing
@landonf
landonf / watchdog.c
Last active September 26, 2025 18:42
Enables OS X's hardware watchdog.
#include <CoreFoundation/CoreFoundation.h>
#include <IOKit/IOKitLib.h>
#include <dispatch/dispatch.h>
// 5 minutes
#define TIMEOUT (5*60)
bool reset_watchdog (int timeout) {
io_service_t service = IOServiceGetMatchingService(kIOMasterPortDefault, IOServiceMatching("IOWatchDogTimer"));
if (service == IO_OBJECT_NULL) {
@natevw
natevw / psc↔pdx|sea transportation schedules.md
Last active August 30, 2018 21:24
Direct connections available between the Tri-Cities and Portland or Seattle, with approximate prices as of 2013 April.

All price estimates are for one-way. Car and train and bus return doubles price, air has significant discount for round trip.

TODO: For longer trips it can be a little cheaper to ride Amtrak by signing up for rewards, buying miles, then booking tickets; haven't calculated cost for Portland yet.

PSC ↔ PDX

Car $26

Downtown Portland is about 215 miles from the Tri-Cities and takes about 3h30m. Cost calculated with gas at $3.69/gal and fuel economy averaging 30mpg (i.e. 12.3¢/mile).

@johnjohndoe
johnjohndoe / delicious.js
Last active July 25, 2017 13:49
Bookmarklet of del.icio.us. Latest version available at https://del.icio.us/tools. This Gist tracks the changes made to the script. Javascript is formatted via http://jsbeautifier.org. - The authors of the script are the guys from del.icio.us.
javascript: (function($) {
var bookmarklet = document.getElementById(
'DELI_save_link_slidedown');
if (bookmarklet) {
$('#DELI_mist').show();
$('#DELI_save_link_slidedown').slideDown('normal');
return
};
if (!window.jQuery) {
node = document.createElement('SCRIPT');
@max-mapper
max-mapper / readme.md
Last active June 3, 2020 00:31
automatically scan for and join open internet enabled wifi networks on linux using node.js (tested on raspberry pi raspbian)
@rodw
rodw / pid-file-daemon.sh
Created August 14, 2012 01:53
Bash script that uses a PID file to add daemon-like start/stop/status behavior to an arbitrary program.
#!/bin/bash
# Uses a PID file to add daemon-like behavior to an arbitrary program.
################################################################################
usage() {
echo "Usage: `basename $0` PROGRAM {start|stop|restart|force-stop|force-restart|status} [PIDFILE|PID]" >&2
echo "Where: PROGRAM is an executable file." >&2
echo " PIDFILE is the file that contains (or will contain) the PID." >&2
echo " PID is a process id to use in place of a PIDFILE." >&2
}