Skip to content

Instantly share code, notes, and snippets.

@TomConlin
Last active March 29, 2018 23:48
Show Gist options
  • Select an option

  • Save TomConlin/4392fc172dc2023bda7cf6f0f5d0f280 to your computer and use it in GitHub Desktop.

Select an option

Save TomConlin/4392fc172dc2023bda7cf6f0f5d0f280 to your computer and use it in GitHub Desktop.
CELL
pub
species
disease
sex
age
category/type
anatomy
marker/allele
xmlstarlet el -a cellosaurus.xml | sort -u > cellosaurus.xpath
xpath2dot.awk cellosaurus.xpath > cellosaurus.gv
dot-T png cellosaurus.gv > cellosaurus.png
wc -l < cellosaurus.xpath
119
###############################################################
# cell line identifier
Cellosaurus/cell-line-list/cell-line/accession-list
Cellosaurus/cell-line-list/cell-line/accession-list/accession
Cellosaurus/cell-line-list/cell-line/accession-list/accession/@type
where @type=primary
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/accession-list/accession[@type='primary']" \
cellosaurus.xml | cut -f1 -d '_' | sort | uniq -c
101,528 CVCL
# unique?
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/accession-list/accession[@type='primary']" \
cellosaurus.xml | sort -u| wc -l
101,528
-----------------------------------------------------
Cellosaurus/cell-line-list/cell-line/
@category=
@sex=
@age=
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/@category" \
cellosaurus.xml | sort | uniq -c | sort -nr
43853 Transformed cell line
18284 Cancer cell line
13479 Embryonic stem cell
8553 Finite cell line
7234 Induced pluripotent stem cell
4264 Hybridoma
4123 Spontaneously immortalized cell line
596 Hybrid cell line
307 Telomerase immortalized cell line
286 Conditionally immortalized cell line
187 Undefined cell line type
170 Adult stem cell
135 Factor-dependent cell line
57 Stromal cell line
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/@sex" \
cellosaurus.xml | sort | uniq -c | sort -nr
46680 Male
38148 Female
4041 Sex unspecified
87 Sex ambiguous
37 Mixed sex
# yuck
# try with numerals and ranges removed
# Years, Months, Weeks, Days
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/@age" \
cellosaurus.xml | tr -d "0123456789-" | sort | uniq -c | sort -nr
39066 Y
14163 Age unspecified
13536 Blastocyst stage
1964 Adult
1041 YM
655 M
645 Fetus
632 FW
505 D
299 Embryo
166 &lt;M
123 W
91 Children
89 Third instar larval stage
82 Embryonic stage
78 &lt;D
57 FD
39 Late embryonic stage
28 Pupal stage
28 MW
26 Dorsal closure stage
21 Larva
15 FM
14 Neonate larva
13 First instar larval stage
11 &lt;Y
9 Yearling
8 Juvenile
8 &gt;Y
7 &gt;FD
7 Early tailbud stage
6 Tadpole stage
6 Extended germ band stage
6 Embryo/neonate larva
5 Prenatal stage
5 Adult frog stage
4 Moulting nymph
4 Fawn
3 Newborn
3 Morula stage
3 Fry
3 Adult stage
2 Somite stage
2 Prepubertal
2 Moulting larva
2 MD
2 Larval stage
2 Fingerling
2 Fifth instar larval stage
2 Blastula stage
1 Prim stage
1 NF stage
1 Late pupal stage
1 Fourth instar larval stage
1 Embryonic stage
1 Early extended germ band stage
1 Calf
1 Blastula high stage
# would like a staging ontology,
# species
Cellosaurus/cell-line-list/cell-line/species-list/cv-term
Cellosaurus/cell-line-list/cell-line/species-list/cv-term/@accession
Cellosaurus/cell-line-list/cell-line/species-list/cv-term/@terminology
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/species-list/cv-term" \
cellosaurus.xml | sort | uniq -c | sort -nr
74534 Homo sapiens
19137 Mus musculus
1908 Rattus norvegicus
1580 Cricetulus griseus
571 Canis lupus familiaris
357 Gallus gallus
272 Bos taurus
244 Drosophila melanogaster
230 Pan troglodytes
183 Sus scrofa
178 Felis catus
157 Mesocricetus auratus
109 Oryctolagus cuniculus
86 Macaca mulatta
69 Chlorocebus aethiops
62 Ovis aries
56 Equus caballus
... and lots more
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/species-list/cv-term" \
cellosaurus.xml | sort -u | wc -l
590
(check if we want all 590 species ... most likely do)
# diseases
Cellosaurus/cell-line-list/cell-line/disease-list
Cellosaurus/cell-line-list/cell-line/disease-list/cv-term
Cellosaurus/cell-line-list/cell-line/disease-list/cv-term/@accession
Cellosaurus/cell-line-list/cell-line/disease-list/cv-term/@terminology
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/disease-list/cv-term" \
cellosaurus.xml | sort | uniq -c | sort -nr
4367 Type 2 diabetes mellitus
2789 Parkinson's disease
1889 Amyotrophic lateral sclerosis
1848 Melanoma
1497 Huntington's disease
1021 Seizure disorder
1018 Ischemic cerebrovascular accident
535 Neuroblastoma
496 Lung adenocarcinoma
489 Small cell lung carcinoma
474 Intracranial aneurysm
431 Human papillomavirus-related endocervical adenocarcinoma
424 Glioblastoma
416 Osteosarcoma
410 Subarachnoid hemorrhage
410 Colon adenocarcinoma
314 Dilated cardiomyopathy
287 Alzheimer's disease
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/disease-list/cv-term" \
cellosaurus.xml | sort -u | wc -l
1617
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/disease-list/cv-term/@accession" \
cellosaurus.xml | cut -f1 -d '_' | sort | uniq -c | sort -nr | head
4367 C26747
2789 C26845
1889 C34373
1848 C3224
1497 C82342
1021 C3020
1018 C95802
535 C3270
496 C3512
490 C4917
# seem to be NCIt terms (keep as is & let clique leader translate)
# anatomy? from their faq.
```
-Tissue: Not yet possible.
We plan to add the tissue of origin using Uberon as the standardization resource,
but this will not take place until 2018.
```
# markers/alleles
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/@conflict
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/@id
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data/alleles
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/@id" \
cellosaurus.xml | sort | uniq -c | sort -nr | head
5393 vWA
5387 Amelogenin
5343 D7S820
5332 D5S818
5331 D13S317
5298 TH01
5264 CSF1PO
5263 TPOX
5068 D16S539
2427 FGA
...
pretty short list, only 41 markers
xmlstarlet sel -t -v "//Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data/alleles" \
cellosaurus.xml | sort | uniq -c | sort -nr | head
3814 11,12
3699 11
3344 X
3323 12
2117 8,11
2103 8
2095 X,Y
2065 10,11
1765 10,12
1607 9
...
hmmm, looks more like chromosomes to me
prefers "alleles" were per chromosome
#############################################################
# pubs
Cellosaurus/publication-list/publication/xref-list/xref
Cellosaurus/publication-list/publication/xref-list/xref/@accession
Cellosaurus/publication-list/publication/xref-list/xref/@category
Cellosaurus/publication-list/publication/xref-list/xref/@database
Cellosaurus/publication-list/publication/xref-list/xref/url
# category
26,054 Reference resources
# database
xmlstarlet sel -t -v "//Cellosaurus/publication-list/publication/xref-list/xref/@database" \
cellosaurus.xml | sort | uniq -c | sort -nr
13584 PubMed
11348 DOI
736 Patent
386 CelloPub
# accession
all but one (10.1007/BF01753729) is unique
xmlstarlet sel -t -v "//Cellosaurus/publication-list/publication/xref-list/xref/@accession" \
cellosaurus.xml | sort | uniq -c | wc -l
26,053
#############################################################
curl -s https://gist.githubusercontent.com/TomConlin/4392fc172dc2023bda7cf6f0f5d0f280/raw/b3dc6650bcc8092b3439f6754adcacf5f5237258/cellosaurus.xpath| xpath2dot.awk -v ORIENT="UD" | dot -T png > cellosaurus_xpath.png
Cellosaurus
Cellosaurus/cell-line-list
Cellosaurus/cell-line-list/cell-line
Cellosaurus/cell-line-list/cell-line/accession-list
Cellosaurus/cell-line-list/cell-line/accession-list/accession
Cellosaurus/cell-line-list/cell-line/accession-list/accession/@type
Cellosaurus/cell-line-list/cell-line/@age
Cellosaurus/cell-line-list/cell-line/@category
Cellosaurus/cell-line-list/cell-line/comment-list
Cellosaurus/cell-line-list/cell-line/comment-list/comment
Cellosaurus/cell-line-list/cell-line/comment-list/comment/@category
Cellosaurus/cell-line-list/cell-line/comment-list/comment/cv-term
Cellosaurus/cell-line-list/cell-line/comment-list/comment/cv-term/@accession
Cellosaurus/cell-line-list/cell-line/comment-list/comment/cv-term/@terminology
Cellosaurus/cell-line-list/cell-line/comment-list/comment/method
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref/@accession
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref/@category
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref/@database
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref/property-list
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref/property-list/property
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref/property-list/property/@name
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref/property-list/property/@value
Cellosaurus/cell-line-list/cell-line/comment-list/comment/xref-list/xref/url
Cellosaurus/cell-line-list/cell-line/derived-from
Cellosaurus/cell-line-list/cell-line/derived-from/cv-term
Cellosaurus/cell-line-list/cell-line/derived-from/cv-term/@accession
Cellosaurus/cell-line-list/cell-line/derived-from/cv-term/@terminology
Cellosaurus/cell-line-list/cell-line/disease-list
Cellosaurus/cell-line-list/cell-line/disease-list/cv-term
Cellosaurus/cell-line-list/cell-line/disease-list/cv-term/@accession
Cellosaurus/cell-line-list/cell-line/disease-list/cv-term/@terminology
Cellosaurus/cell-line-list/cell-line/name-list
Cellosaurus/cell-line-list/cell-line/name-list/name
Cellosaurus/cell-line-list/cell-line/name-list/name/@type
Cellosaurus/cell-line-list/cell-line/reference-list
Cellosaurus/cell-line-list/cell-line/reference-list/reference
Cellosaurus/cell-line-list/cell-line/reference-list/reference/@resource-internal-ref
Cellosaurus/cell-line-list/cell-line/same-origin-as
Cellosaurus/cell-line-list/cell-line/same-origin-as/cv-term
Cellosaurus/cell-line-list/cell-line/same-origin-as/cv-term/@accession
Cellosaurus/cell-line-list/cell-line/same-origin-as/cv-term/@terminology
Cellosaurus/cell-line-list/cell-line/@sex
Cellosaurus/cell-line-list/cell-line/species-list
Cellosaurus/cell-line-list/cell-line/species-list/cv-term
Cellosaurus/cell-line-list/cell-line/species-list/cv-term/@accession
Cellosaurus/cell-line-list/cell-line/species-list/cv-term/@terminology
Cellosaurus/cell-line-list/cell-line/str-list
Cellosaurus/cell-line-list/cell-line/str-list/marker-list
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/@conflict
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/@id
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data/alleles
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data/source-list
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data/source-list/reference-list
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data/source-list/reference-list/reference
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data/source-list/reference-list/reference/@resource-internal-ref
Cellosaurus/cell-line-list/cell-line/str-list/marker-list/marker/marker-data-list/marker-data/source-list/source
Cellosaurus/cell-line-list/cell-line/str-list/source-list
Cellosaurus/cell-line-list/cell-line/str-list/source-list/reference-list
Cellosaurus/cell-line-list/cell-line/str-list/source-list/reference-list/reference
Cellosaurus/cell-line-list/cell-line/str-list/source-list/reference-list/reference/@resource-internal-ref
Cellosaurus/cell-line-list/cell-line/web-page-list
Cellosaurus/cell-line-list/cell-line/web-page-list/url
Cellosaurus/cell-line-list/cell-line/xref-list
Cellosaurus/cell-line-list/cell-line/xref-list/xref
Cellosaurus/cell-line-list/cell-line/xref-list/xref/@accession
Cellosaurus/cell-line-list/cell-line/xref-list/xref/@category
Cellosaurus/cell-line-list/cell-line/xref-list/xref/@database
Cellosaurus/cell-line-list/cell-line/xref-list/xref/property-list
Cellosaurus/cell-line-list/cell-line/xref-list/xref/property-list/property
Cellosaurus/cell-line-list/cell-line/xref-list/xref/property-list/property/@name
Cellosaurus/cell-line-list/cell-line/xref-list/xref/property-list/property/@value
Cellosaurus/cell-line-list/cell-line/xref-list/xref/url
Cellosaurus/copyright
Cellosaurus/header
Cellosaurus/header/description
Cellosaurus/header/release
Cellosaurus/header/release/@nb-cell-lines
Cellosaurus/header/release/@nb-publications
Cellosaurus/header/release/@updated
Cellosaurus/header/release/@version
Cellosaurus/header/terminology-list
Cellosaurus/header/terminology-list/terminology
Cellosaurus/header/terminology-list/terminology/@description
Cellosaurus/header/terminology-list/terminology/@name
Cellosaurus/header/terminology-list/terminology/@source
Cellosaurus/header/terminology-list/terminology/url
Cellosaurus/header/terminology-name
Cellosaurus/publication-list
Cellosaurus/publication-list/publication
Cellosaurus/publication-list/publication/author-list
Cellosaurus/publication-list/publication/author-list/person
Cellosaurus/publication-list/publication/author-list/person/@name
Cellosaurus/publication-list/publication/@city
Cellosaurus/publication-list/publication/@country
Cellosaurus/publication-list/publication/@date
Cellosaurus/publication-list/publication/editor-list
Cellosaurus/publication-list/publication/editor-list/person
Cellosaurus/publication-list/publication/editor-list/person/@name
Cellosaurus/publication-list/publication/@firstpage
Cellosaurus/publication-list/publication/@institute
Cellosaurus/publication-list/publication/@internal-id
Cellosaurus/publication-list/publication/@lastpage
Cellosaurus/publication-list/publication/@name
Cellosaurus/publication-list/publication/@publisher
Cellosaurus/publication-list/publication/title
Cellosaurus/publication-list/publication/@type
Cellosaurus/publication-list/publication/@volume
Cellosaurus/publication-list/publication/xref-list
Cellosaurus/publication-list/publication/xref-list/xref
Cellosaurus/publication-list/publication/xref-list/xref/@accession
Cellosaurus/publication-list/publication/xref-list/xref/@category
Cellosaurus/publication-list/publication/xref-list/xref/@database
Cellosaurus/publication-list/publication/xref-list/xref/url
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment