Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active June 24, 2025 11:12
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
using OpenTelemetry;
using OpenTelemetry.Logs;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
var appResourceBuilder = ResourceBuilder.CreateDefault()
.AddService(serviceName: Telemetry.ServiceName, serviceVersion: Telemetry.ServiceVersion);
#!/usr/bin/env bash
# on ubuntu 18.04
cat << EOF >/etc/apt/source.list
deb http://archive.ubuntu.com/ubuntu bionic main multiverse restricted universe
deb http://archive.ubuntu.com/ubuntu bionic-security main multiverse restricted universe
deb http://archive.ubuntu.com/ubuntu bionic-updates main multiverse restricted universe
EOF
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
cat << EOF > /etc/apt/sources.list.d/clang.list
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\SOFTWARE\Microsoft\PCHC]
"UpgradeEligibility"=dword:00000001
[HKEY_LOCAL_MACHINE\SYSTEM\Setup\LabConfig]
"BypassTPMCheck"=dword:00000001
"BypassSecureBootCheck"=dword:00000001
"BypassRAMCheck"=dword:00000001
"BypassStorageCheck"=dword:00000001
@davidfowl
davidfowl / .NET6Migration.md
Last active November 18, 2025 22:14
.NET 6 ASP.NET Core Migration
@ejlp12
ejlp12 / eks_best_practice.md
Last active September 16, 2025 23:36
EKS Best Practice

Please check https://aws.github.io/aws-eks-best-practices/ for more comprehensive EKS best practice!

Architecture

  • Think about multi-tenancy, isolation for different environment or different workload
    • Isolation at account level using AWS organization
    • Isolation at the network layer ie. different VPC & different cluster
    • Use different Nodes Group (Node pool) for different purpose/category e.g. create dedicated node groups for Operational tools such as CI/CD tool, Monitoring tool, Centralize logging system.
    • Separate namespace for different workload

Reliability | Principles

@yellow1912
yellow1912 / nginx.toml
Last active July 15, 2025 09:33
Sample Vector configuration for nginx
# be careful, you will gets tons of logs
# you may want to skip access logs or use sampling only here
[sources.nginx_access_logs]
type = "file"
include = ["/var/log/sites/*/*/*/*/nginx/access.log"] # supports globbing
ignore_older = 86400 # 1 day
# I'm configuring my nginx to use json logs, if you use another logging format you need to change this
[transforms.nginx_access_logs_json]
type = "json_parser"
@tsumarios
tsumarios / portscanner.md
Created July 2, 2020 09:54
Browser-based Port Scanner. Tested on Firefox.
@ArturKlauser
ArturKlauser / check-qemu-binfmt.sh
Last active February 13, 2023 16:53
Building Multi-Architecture Docker Images With Buildx
#!/bin/bash
# (c) 2020 Artur.Klauser@computer.org
# SPDX-License-Identifier: Apache-2.0 OR MIT
#
# This script checks if all software requirements are met in a Linux environment
# in order to use 'docker buildx' to build multi-architecture images.
# For more information see:
# https://nexus.eddiesinentropy.net/2020/01/12/Building-Multi-architecture-Docker-Images-With-Buildx/
function error() {