new-system-setup.md

Steps to Setup a New Machine (and more)

• Homebrew (macOS only)
• iTerm2 (macOS only)
• Shell
  • ZSH
  • Starship Prompt
  • fzf, bat, and fd
• Vim
• SSH
  • Mutagen: Persistent Port Forwarding
• Python
  • Miniconda Mamba
    • Sample Envirmoment Setup
  • PDB Improvements
  • Useful Utilities
• Git
• Java
• Javascript
• Rust
• IDEs & REPLs
  • VS Code
    • code-server
  • JetBrains
    • Deployment
    • CLion
  • Ammonite
  • Almond
• Applications

Homebrew (macOS only)

Follow steps in Install section here. May need to add /opt/homebrew/bin to PATH.

Use brew to install gcc, tree, fswatch, and tmux: brew install gcc tree fswatch tmux.

Note: clang doesn't include bits/stdc++.h. Manually download it and put it at this path /usr/local/include/bits/stdc++.h.

iTerm2 (macOS only)

1. Follow Install section here.
   • Make sure to create a new profile and set it as default. Subsequent modifications will be done to this profile.
2. Download Material Themes from here and here and set theme to material-theme.
3. Set terminal theme to Solarized Dark Theme (already installed).
4. Set Key bindings
   1. Enable Natural Text Editing for new profile (ref):
      1. Go to Preferences... > Profiles > Keys
      2. Press Load Preset...
      3. Select Natural Text Editing
   2. Delete Key bindings for ⌘← and ⌘→.
5. Powerline Fonts for oh-my-zsh themes.
   1. Install Powerline Fonts.
   2. Set terminal font to Meslo LG S or some other powerline fonts.
6. Modern Titlebar: Preferences > Appearance > General > Select Minimal from Theme dropdown.
7. Enable Shell Integration.

Shell

ZSH

1. Install zsh and oh-my-zsh as described here.
2. Configure theme: ZSH_THEME="agnoster".
3. Configure plugins

   1. Clone plugins

      git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ~/.oh-my-zsh/custom/plugins/zsh-syntax-highlighting &&
      git clone https://github.com/zsh-users/zsh-autosuggestions ~/.oh-my-zsh/custom/plugins/zsh-autosuggestions

   2. Set in ~/.zshrc

      plugins=(z git colored-man-pages colorize osx zsh-syntax-highlighting zsh-autosuggestions extract)

Starship Prompt

Install Starship prompt.

1. On mac: brew install starship. On linux: curl -fsSL https://starship.rs/install.sh | bash

2. Add to .zshrc:

   eval "$(starship init zsh)"

3. Create config file ~/.config/starship.toml with contents:

   # Replace the "❯" symbol in the prompt with "❯❯ "
   [character]                            # The name of the module we are configuring is "character"
   #format = "[❯❯] "
   success_symbol = "[❯❯](bold green) "
   error_symbol = "[❯❯](bold red) "
   
   # Disable the AWS module.
   [aws]
   disabled = true

fzf, bat, and fd

1. Install fzf, bat, and fd

   1. With Homebrew:

      brew install fzf bat fd
      
      # To install useful key bindings and fuzzy completion:
      $(brew --prefix)/opt/fzf/install

   2. Ubuntu:

      git clone --depth 1 https://github.com/junegunn/fzf.git ~/.fzf
      ~/.fzf/install
      sudo apt install bat fd-find
      ln -s $(which fdfind) ~/.local/bin/fd
      ln -s $(which batcat) ~/.local/bin/bat

   3. Without sudo: This will require Rust/Cargo to be installed.

      git clone --depth 1 https://github.com/junegunn/fzf.git ~/.fzf
      ~/.fzf/install
      curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh  # install `rustup` for `cargo`
      source "$HOME/.cargo/env"
      cargo install --locked bat
      cargo install fd-find

2. Add the following to ~/.zshrc to configure fzf to use bat and fd and add useful keybindings (ref):

   export FZF_DEFAULT_OPTS="
   --layout=reverse
   --info=inline
   --multi
   --preview-window=:hidden
   --ansi
   --select-1
   --preview '([[ -f {} ]] && (bat --style=numbers --color=always {} || cat {})) || ([[ -d {} ]] && (tree -C {} | less)) || echo {} 2> /dev/null | head -200'
   --color='hl:148,hl+:154,pointer:032,marker:010,bg+:237,gutter:008'
   --prompt='∼ ' --pointer='▶' --marker='✓'
   --bind '?:toggle-preview'
   --bind 'ctrl-a:select-all'
   --bind 'ctrl-y:execute-silent(echo {+} | pbcopy)'
   --bind 'ctrl-e:execute(echo {+} | xargs -o vim)'
   --bind 'ctrl-v:execute(code {+})'
   "
   FD_OPTIONS="--hidden --follow --color=always --exclude '.git' --exclude 'node_modules'"
   export FZF_DEFAULT_COMMAND="fd $FD_OPTIONS"
   # CTRL-T's command
   export FZF_CTRL_T_COMMAND="fd $FD_OPTIONS"
   # like normal z when used with arguments but displays an fzf prompt when used without.
   unalias z 2> /dev/null
   z() {
       [ $# -gt 0 ] && _z "$*" && return
       cd "$(_z -l 2>&1 | fzf --height 40% --nth 2.. --reverse --inline-info +s --tac --query "${*##-* }" | sed 's/^[0-9,.]* *//')"
   }    

Vim

1. Setup Pahthogen.vim following the steps here.

   mkdir -p ~/.vim/autoload ~/.vim/bundle && git clone git@github.com:tpope/vim-pathogen.git ~/.vim/autoload/pathogen.vim

2. Create ~/.vimrc:

   execute pathogen#infect()
   syntax on
   filetype plugin indent on
   
   " Redfine <kbd>⇥ Tab</kbd> to insert 4 spaces  (https://stackoverflow.com/questions/1878974/redefine-tab-as-4-spaces)
   " show existing tab with 4 spaces width
   set tabstop=4
   
   " when indenting with '>', use 4 spaces width
   set shiftwidth=4
   
   " On pressing tab, insert 4 spaces
   set expandtab
   
   set softtabstop=0 smarttab

3. Install plugins.

   git clone https://github.com/tpope/vim-sensible.git ~/.vim/bundle/vim-sensible
   git clone https://github.com/elzr/vim-json ~/.vim/bundle/vim-json
   git clone https://github.com/Raimondi/delimitMate.git ~/.vim/bundle/delimitMate
   git clone https://github.com/vim-airline/vim-airline ~/.vim/bundle/vim-airline

SSH

1. Follow this for appropriate OS to create a new key pair and add it to ssh-agent.
2. Configure ~/.ssh/config with all remote hosts.
   • basic examples
   • in-depth tutorial
3. Copy public key to the ~/.ssh/known-hosts file on all remotes to enable password-less ssh.

Mutagen: Persistent Port Forwarding

Mutagen can be used to create persistent port-forwarding tunnels between machines.

Installation: brew install mutagen-io/mutagen/mutagen

Example Usage: mutagen forward create --name=code-server tcp:127.0.0.1:8080 uci-ava-s0:tcp:127.0.0.1:8080 - connect local port 8080 to remote port 8080.

Python

Miniconda Mamba

1. Download appropriate miniconda mambaforge installer and install.

   wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-MacOSX-arm64.sh
   bash Mambaforge-MacOSX-arm64.sh

2. Create environment: mamba create -n <env_name> python=3.9

3. Search packages: Anaconda, PyPi

4. Install packages

Sample Envirmoment Setup

# wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-MacOSX-arm64.sh
# bash Mambaforge-MacOSX-arm64.sh
mamba create -n dl python=3.9
mamba activate dl
dev=(jupyter jupyterlab pylint voila rich gpustat)        # dev tools
app=(flask typer streamlit omegaconf hydra-core)                 # app development
ds_ml=(numpy scipy pandas matplotlib seaborn scikit-learn hyperopt nltk spacy) # datascience, machine learning
numpy=(numexpr bottleneck)             # numpy, pandas acceleration
prog=(sortedcontainers typeguard cattrs anytree inflection more-itertools munch simple-parsing)    # python libraries
misc=(gdown)

mamba install $dev $app $ds_ml $numpy $prog $misc

# PyTorch
mamba install pytorch torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidia
# Use this instead of above for cpu-only version (eg. on Mac)
# mamba install -c pytorch pytorch torchvision torchaudio torchtext

# Tensorflow
# mamba install -c anaconda cudatoolkit=10.2 tensorflow-gpu keras
# pip install t5

# Jax
# pip install --upgrade jax jaxlib

mamba install transformers datasets tokenizers   # huggingface
mamba install -c comet_ml comet_ml               # expriment tracking
pip install 'ray[tune]'                                         # optimization
pip install allennlp                                            # NLP
mamba install openai
# pip install codalab -U --user                                   # experiment management

Note: allennlp is being installed using pip because conda-forge doesn't yet have a install for MacOS for it (allennlp issue) and Linux install wasn't updated at the time of writing.

Ensure that the installed packages import fine with:

import rich
import numpy, scipy, pandas
import matplotlib, seaborn
import nltk, sklearn
import sortedcontainers typeguard cattrs anytree inflection
import flask, streamlit
import comet_ml
import torch, torchtext, torchvision, torchaudio
import transformers, datasets, tokenizers
import allennlp
import hyperopt
from ray import tune

PDB Improvements

1. Install ipdb and set it as default:

mamba install ipdb
echo 'export PYTHONBREAKPOINT=ipdb.set_trace' >> ~/.zshrc

3. Copy the .pdbrc.py from here to ~/.pdbrc.py.

Useful Utilities

1. Dataclass on steroids: parameters.py
2. Typer Dataclass: typer-dataclass.py
3. Rich Logging: logging.py
4. Rich progress bars: track.py

Git

Configure name and email globally:

git config --global user.name "Shivanshu Gupta"
git config --global user.email "shivanshugupta1995@gmail.com"

Java

1. Install jEnv to manage JDKs.
2. Download JDK from here.

Javascript

1. Install nvm, node and npm.

   curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash     # install nvm
   nvm install node    # install node (and npm)

Rust

1. Install rustup for cargo: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs -sSf | sh

IDEs & REPLs

Purpose	Small Projects	Large Projects	Shell/REPL	Notebook
Text	Sublime Text, VSCode	-	-	-
Python	VSCode	PyCharm	IPython	Jupyter Lab
Scala	IntelliJ	IntelliJ	Ammonite	Almond + Jupyter Lab
Julia	VSCode	VSCode	julia	Jupyter Lab
C/C++	VSCode	CLion	-	-

VS Code

Follow the steps here to use the Settings Sync to sync all extensions, settings etc.

code-server

Use code-server to run VS Code on any server and access locally from browser:

1. Install code-server on remote server (more details here):

   curl -fsSL https://code-server.dev/install.sh | sh -s -- --method=standalone

2. Run code-server on remote server (more details here):

   • Through VPN
     • On server (eg. ava-s0): code-server --port=8080
     • Locally: go to http://uci-ava-s0.ics.uci.edu:8080
   • Without VPN
     • Option 1 (using port-forwarding):
       • On server: code-server --port=8080
       • Locally: Create a persistent port-forward using mutagen
     • Option 2: using persistent link
       • On server: code-server --link
       • Locally: go to generated url

3. Configurations are in ~/.config/code-server/config.yaml. Can be overridden by the arguments to code-server command.

JetBrains

Deployment

1. Configure remotes with appropriate mappings and exclusions
2. Configure to upload on save TODO

CLion

1. In case unable to #include <bits/stdc++.h> only in Clion, follow first step here.

Ammonite

brew install ammonite-repl

Almond

Follow steps here to install.

Applications

1. Utilities
   1. Rectangle - brew cask install rectangle
   2. CloudApp
   3. Cyberduck - sftp and more. (On mac, just use iTerm2 Shell Integrations if only downloading single files.
   4. Fantastical 2.5.16 Crack
      1. Open the downloaded dmg.
      2. Open the dmg in Manual install directory.
      3. Copy the Fantastical 2.app file to ~/Downloads.
      4. Right click Fantastical 2.app and Open.
   5. AltTab
2. Productivity
   1. Roam for Research
   2. Notion
   3. Raycast
   4. Markdown: Typora, MarkText, Zettlr
   5. Microsoft Office
   6. OmniGraffle
3. IDEs & Developer Tools
   1. VS Code
   2. Jetbrains: IntelliJ, CLion, PyCharm, JetBrains Toolbox
   3. Jupyter Lab
   4. Ammonite REPL
   5. TunnelBlick VPN
   6. Docker
4. Communication: Slack, Zoom, Bluejeans, Outlook, Discord, Wire, Skype
5. Academics
   1. Zotero
   2. MathPix Snipping Tool
   3. Latex: TexShop, Latexit
6. Media
   1. Photoscape - Photo Editor, Photo Viewer etc.
   2. IINA - Media Player