This is how I did it, it's definitely not conventional but I didn't find any post about this. Try at your own risk!
- "python setup.py install"
- Check the egg packages with "pip list"
- Convert the egg packages to wheel (wheel convert )
- Extract the wheel
- Check the dist-info folder, and check if the folder names are proper. If you need to change it, change RECORD and top_level.txt in dist-info folder.
- Copy the main folder (detectron2), additional folder (tools, renamed to detectron2.tools), and the dist-info (detectron2-0.6.dist-info)