paul-krohn

Preamble

There is a longstanding issue/missing feature/bug with sockets on Docker on macOS; it may never work; you'll need to use a network connection between Docker containers and X11 on macOS for the foreseeable future.

I started from this gist and made some adjustments:

• the volume mappings aren't relevant/used, due to the socket issue above.
• this method only allows X11 connections from your Mac, not the entire local network, which would include everyone on the café/airport WiFi.
• updated to include using the host.docker.internal name for the the container host, instead.
• you have to restart XQuartz after the config change.

sparkydogX

import os
import torch
from tqdm import tqdm
import time

# declare which gpu device to use
cuda_device = '0'

def check_mem(cuda_device):
    devices_info = os.popen('"/usr/bin/nvidia-smi" --query-gpu=memory.total,memory.used --format=csv,nounits,noheader').read().strip().split("\n")

machinaut

FROM ubuntu:18.04

# Install python and utils
RUN apt-get update && apt-get install -y python3-pip curl unzip \
    libosmesa-dev libglew-dev patchelf libglfw3-dev

# Download mujoco
RUN curl https://www.roboti.us/download/mjpro150_linux.zip --output /tmp/mujoco.zip && \
    mkdir -p /root/.mujoco && \
    unzip /tmp/mujoco.zip -d /root/.mujoco && \

ZijiaLewisLu

In most of deep learning projects, the training scripts always start with lines to load in data, which can easily take a handful minutes. Only after data ready can start testing my buggy code. It is so frustratingly often that I wait for ten minutes just to find I made a stupid typo, then I have to restart and wait for another ten minutes hoping no other typos are made.

In order to make my life easy, I devote lots of effort to reduce the overhead of I/O loading. Here I list some useful tricks I found and hope they also save you some time.

1. use Numpy Memmap to load array and say goodbye to HDF5.

   I used to relay on HDF5 to read/write data, especially when loading only sub-part of all data. Yet that was before I realized how fast and charming Numpy Memmapfile is. In short, Memmapfile does not load in the whole array at open, and only later "lazily" load in the parts that are required for real operations.

Sometimes I may want to copy the full array to memory at once, as it makes later operations

apaszke

from graphviz import Digraph
import torch
from torch.autograd import Variable, Function

def iter_graph(root, callback):
    queue = [root]
    seen = set()
    while queue:
        fn = queue.pop()
        if fn in seen:

Adriel-M

# REINFORCE: Monte Carlo Policy Gradient Implementation
# Learn more from Reinforcement Learning: An Introduction (p271)
# by Sutton & Barto

import tensorflow as tf
import gym
import numpy as np
from gym import wrappers

robdvr

1. Remove qt4,5 poppler, cairo

brew uninstall cairo
brew poppler
brew qt
brew qt5

2. Install qt4 on Sierra & link qt (this will take about 20 mins)

poolio

def _phase_shift(I, r):
    bsize, a, b, c = I.get_shape().as_list()
    X = tf.reshape(I, (bsize, a, b, r, r))
    X = tf.transpose(X, (0, 1, 2, 4, 3))  # bsize, a, b, 1, 1
    X = tf.split(1, a, X)  # a, [bsize, b, r, r]
    X = tf.concat(2, [tf.squeeze(x, [1]) for x in X])  # bsize, b, a*r, r
    X = tf.split(1, b, X)  # b, [bsize, a*r, r]
    X = tf.concat(2, [tf.squeeze(x, [1]) for x in X])  # bsize, a*r, b*r
    return tf.reshape(X, (bsize, a*r, b*r, 1))

blole

#!/usr/bin/env python2
import os
import gym
import sys
import random
import itertools
from time import time
from copy import copy
from math import sqrt, log

bastman

Docker - How to cleanup (unused) resources

Once in a while, you may need to cleanup resources (containers, volumes, images, networks) ...

delete volumes

// see: https://github.com/chadoe/docker-cleanup-volumes

$ docker volume rm $(docker volume ls -qf dangling=true)

$ docker volume ls -qf dangling=true | xargs -r docker volume rm