Re:Novel coronavirus complete genome from the Wuhan outbreak...
by BioGeek on Tuesday January 28, 2020 @09:08AM
(#22169456)
Using Biopython. Note that the search query that I am using currently returns 70605 results, so you might want to tweak it fit your needs.
from Bio import Entrez
import time
from urllib.error import HTTPError
DB = 'nucleotide'
QUERY = '("pneumoviridae"[Organism] OR "Coronaviridae"[Organism])'
Entrez.email = '[email protected]'
handle = Entrez.esearch(db=DB, term=QUERY, rettype='fasta')
record = Entrez.read(handle)
handle = Entrez.esearch(db=DB, term=QUERY, retmax=count, rettype='fasta')
record = Entrez.read(handle)
id_list = record['IdList']
count = len(id_list)
post_xml = Entrez.epost(DB, id=",".join(id_list))
search_results = Entrez.read(post_xml)
webenv = search_results['WebEnv']
query_key = search_results['QueryKey']
batch_size = 200
with open('viruses.fasta', 'w') as out_handle:
for start in range(0, count, batch_size):
end = min(count, start+batch_size)
print(f"Going to download record {start+1} to {end}")
attempt = 0
success = False
while attempt < 3 and not success:
attempt += 1
try:
fetch_handle = Entrez.efetch(db=DB, rettype='fasta',
retstart=start, retmax=batch_size,
webenv=webenv, query_key=query_key)
success = True
except HTTPError as err:
if 500 <= err.code <= 599:
print(f"Received error from server {err}")
print("Attempt {attempt} of 3")
time.sleep(15)
else:
raise
data = fetch_handle.read()
fetch_handle.close()
out_handle.write(data)
Re:Novel coronavirus complete genome from the Wuhan outbreak...
by Smerity on Tuesday January 28, 2020 @02:33PM
(#22173196)
Champion! Thank you =]
Re:Novel coronavirus complete genome from the Wuhan outbreak...
by BioGeek on Wednesday January 29, 2020 @04:17AM
(#22178768)
There is a small error in the code. The variable `count` should be defined on line 11 like:
count = int(record["Count"])
en the appearance on line 15 should be removed.
Re:Novel coronavirus complete genome from the Wuhan outbreak...
by jerven on Tuesday January 28, 2020 @05:10AM
(#22167981)
https://viralzone.expasy.org/8676
Is a good start.
I work in language modeling and want to see about using those unsupervised / self-supervised methods for genome annotation / phylogenetic tree construction.
Even if it's more of a curiousity than a useful tool I have experience with small datasets, most recently focused on character level language modeling on ~90MB of training data, so if I can get (90MB / (29 kilobases * 2 bits per base) =) approximately 12,000 related samples I should be able to at least make a dataset out of it.