dbGaP study info dump json file is available on ftp

[jameslhao]

The following is the email sent to David. Put it over here for the record.
James

---

From: James L. Hao [email protected]
Date: Wed, Aug 16, 2017 at 8:04 PM
Subject: Re: empty files ftpDownload dbgapr
To: "McGaughey, David (NIH/NEI) [E]" [email protected]

Hi David,

1. The dbGaP study info dump json is available now through the following ftp. The
   ftp://ftp.ncbi.nlm.nih.gov/dbgap/r-tool/public_datadump/

You may go through the sample file again. It includes 4 studies. 2 of them are root studies, another 2 are sub-studies.
sample_dbgap_study_info_dump_pretty.json

The fields 'is_root', 'has_child', and 'has_parent' can help to identify parent-child relationship.

The sample and subject count of chip info will be added later.

2. I looked into the empty ftp files issue of phs000803.v1.p1. It turns out that the respective database tables are not loaded. It happens occupationally because of all kind of reasons. You may simply ignore them in this case. It should be populated sometime later in most of cases.

If you search for phs000803 through the Advanced Search (URL below), you will see 0 variable returned, which confirms that the variable related table is indeed empty.
https://www.ncbi.nlm.nih.gov/projects/gapsolr/facets.html

Please do not hesitate to write back if you have any questions.

Keep in touch.

Cheers,
Luning

[seandavi]

Thanks, @jameslhao, for doing all this extra work for the project!

[davemcg]

Thanks @jameslhao, much appreciated

[jameslhao]

Happy to help. Let me know if you guys decide to use the new datadump file on the ftp. Cheers, James

…

On Thu, Aug 17, 2017 at 10:05 AM, Sean Davis ***@***.***> wrote: Thanks, @jameslhao <https://github.com/jameslhao>, for doing all this extra work for the project! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQKZJJ5chFH7K5mR55YGSMZg8j66sRyZks5sZEi4gaJpZM4O6MQU> .

[seandavi]

Hi, @jameslhao.

The json data dump file makes a lot of sense and would, hopefully, be easier to parse across languages, etc. It would be great to systematically (as completely as possible) include the data from the solr page:

https://www.ncbi.nlm.nih.gov/projects/gapsolr/facets.html

The idea would be that data that are available on the public website should be available in a computer-readable form. Let us know if and how we can help.

[jameslhao]

Hi Sean, Thanks for the interest. One of the items on our to do list for the Advanced Search is to add options of returning search results in various formats including json and XML. I will let you know once it is implemented. I am not very sure how about the timing, but it shouldn't be too long. It may speed up the process if you could write a formal request (as a dbGaP user) to [email protected]. Regards, James

…

On Tue, Aug 22, 2017 at 8:51 AM, Sean Davis ***@***.***> wrote: Hi, @jameslhao <https://github.com/jameslhao>. The json data dump file makes a lot of sense and would, hopefully, be easier to parse across languages, etc. It would be great to systematically (as completely as possible) include the data from the solr page: https://www.ncbi.nlm.nih.gov/projects/gapsolr/facets.html The idea would be that data that are available on the public website should be available in a computer-readable form. Let us know if and how we can help. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQKZJLobke6MOKugTYZTZ0VIvTIWq_68ks5sas63gaJpZM4O6MQU> .