Skip to content

Instantly share code, notes, and snippets.

View rockwotj's full-sized avatar
:shipit:

Tyler Rockwood rockwotj

:shipit:
View GitHub Profile
@rockwotj
rockwotj / windowed-epoch-design.md
Created January 10, 2026 18:09
Windowed Epoch Fencing Design for Redpanda Cloud Topics

Windowed Epoch Fencing for Cloud Topics

Background

Current Epoch Fencing Mechanism

The Cloud Topics Producer (CTP) State Machine uses epoch fencing to ensure that writes to cloud topics happen with monotonically increasing cluster epochs, preventing out-of-order writes and maintaining consistency across the distributed system.

Current Architecture:

  • Each produce request uploads data to object storage and receives an epoch
#edit
input:
http_server:
path: /post/chat
allowed_verbs:
- POST
pipeline:
processors:
- openai_chat_completion:
api_key: "${USER_CONFIGURED_OPENAI_API_KEY}"
input:
postgres_cdc:
dsn: "${POSTGRES_DSN}"
stream_snapshot: true
schema: public
tables:
- demo_table
- other_table
slot_name: rp_connect_repl_slot
snapshot_batch_size: 1024
input:
generate:
mapping: |
root = {
"bar": random_int(),
"baz": uuid_v4(),
"foo": uuid_v4(),
"host": nanoid(64),
"host2": nanoid(54),
"id": counter(),
@rockwotj
rockwotj / clang-build.sh
Last active August 9, 2024 13:20
Building Clang From source
cmake -G Ninja -DLLVM_TARGETS_TO_BUILD=X86 \
-DCMAKE_INSTALL_PREFIX=$PWD/install \
-B $PWD/build \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_INCLUDE_TESTS=OFF \
-DLLVM_INCLUDE_EXAMPLES=OFF \
-DLLVM_INCLUDE_UTILS=OFF \
-DLLVM_INCLUDE_DOCS=OFF \
-DLLVM_ENABLE_ZLIB=OFF \
-DLLVM_ENABLE_ZSTD=OFF \
@rockwotj
rockwotj / BUILD
Created April 25, 2024 01:09 — forked from bsilver8192/BUILD
Basics of generating a compile_commands.json file with Bazel
py_binary(
name = 'generate_compile_command',
srcs = [
'generate_compile_command.py',
],
deps = [
'//third_party/bazel:extra_actions_proto_py',
],
)
@rockwotj
rockwotj / NativeJvmElasticSearchTransport.kt
Created October 25, 2022 19:35
A Native JVM transport for elasticsearch using the builtin JDK HttpClient instead of apache
private class NativeJvmTransport(private val config: SearchClient.Config) : ElasticsearchTransport {
private val options = Options(
headers = ImmutableMultimap.builder<String, String>().apply {
val mimeType = Version.VERSION?.let { version -> "application/vnd.elasticsearch+json; compatible-with=${version.major()}" } ?: "application/json"
put("Content-Type", mimeType)
put("Accept", mimeType)
config.apiKey?.let { put("Authorization", "ApiKey $it") }
}.build()
)
<!DOCTYPE html>
<html>
<head>
<title></title>
<style type="text/css">
#root {
display:flex;
flex-direction: column-reverse;
overflow-y: scroll;
height: 300px;
@rockwotj
rockwotj / BUILD.bazel
Created June 22, 2022 03:44
BUILD.bazel for Mimalloc to use in cc_binary(malloc = ...)
cc_library(
name = "mimalloc",
srcs = ["src/static.c"],
textual_hdrs = glob(["src/*.c", "src/*.h"], exclude = ["src/static.c"]),
hdrs = glob(["include/*.h"]),
includes = ["include"],
copts = ["-DMI_MALLOC_OVERRIDE"],
visibility = ["//visibility:public"],
)
@rockwotj
rockwotj / Contents.json
Created June 18, 2022 01:53
iphone svg asset json file
{
"images" : [
{
"filename" : "test.svg",
"idiom" : "iphone"
}
],
"info" : {
"author" : "xcode",
"version" : 1