Browse Source

Move reference fetching code into a sub-library

master
Lucas Verney 7 years ago
parent
commit
c880fe27a8
  1. 2
      .gitmodules
  2. 10
      README.md
  3. 4
      fetch_references.py
  4. 0
      reference_fetcher/__init__.py
  5. 3
      reference_fetcher/arxiv.py
  6. 6
      reference_fetcher/bbl.py
  7. 4
      reference_fetcher/doi.py
  8. 0
      reference_fetcher/opendetex
  9. 0
      reference_fetcher/regex.py
  10. 0
      reference_fetcher/tools.py

2
.gitmodules vendored

@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
[submodule "opendetex"]
path = opendetex
path = reference_fetcher/opendetex
url = https://github.com/Phyks/opendetex

10
README.md

@ -8,19 +8,19 @@ For building `opendetex` (which is a necessary dependency), you will need @@ -8,19 +8,19 @@ For building `opendetex` (which is a necessary dependency), you will need
* Clone this repository: `git clone https://github.com/Phyks/arxiv_metadata`.
* Init submodules (`opendetex`): `git submodule init; git submodule update`.
* Build `opendetex`: `cd opendetex; make`.
* Build `opendetex`: `cd reference_fetcher/opendetex; make`.
* You are ready to go.
## Usage
* `./main.py some_file.bbl` to get a list of DOIs associated to each `\bibitem`.
* `./main.py arxiv_eprint_id` to get a list of DOIs associated to each reference from the provided arXiv eprint.
* `./fetch_references.py some_file.bbl` to get a list of DOIs associated to each `\bibitem`.
* `./fetch_references.py arxiv_eprint_id` to get a list of DOIs associated to each reference from the provided arXiv eprint.
## Example
```
$ ./main.py 1401.2910
$ ./fetch_references.py 1401.2910
```
```
@ -67,7 +67,7 @@ $ ./main.py 1401.2910 @@ -67,7 +67,7 @@ $ ./main.py 1401.2910
```
$ ./main.py /tmp/test.bbl
$ ./fetch_reference.py /tmp/test.bbl
```
```

4
main.py → fetch_references.py

@ -3,8 +3,8 @@ import os @@ -3,8 +3,8 @@ import os
import sys
# Local import
import arxiv
import bbl
from ..reference_fetcher import arxiv
from ..reference_fetcher import bbl
if __name__ == "__main__":

0
reference_fetcher/__init__.py

3
arxiv.py → reference_fetcher/arxiv.py

@ -1,11 +1,12 @@ @@ -1,11 +1,12 @@
"""
This file contains all the arXiv-specific functions.
"""
import bbl
import io
import requests
import tarfile
from . import bbl
def sources_from_arxiv(eprint):
"""

6
bbl.py → reference_fetcher/bbl.py

@ -1,14 +1,14 @@ @@ -1,14 +1,14 @@
"""
This files contains all the functions to deal with bbl files.
"""
import doi
import math
import os
import requests
import subprocess
import regex
import tools
from . import doi
from . import regex
from . import tools
def clean_bibitem(bibitem):

4
doi.py → reference_fetcher/doi.py

@ -3,8 +3,8 @@ This file contains all the DOI-related functions. @@ -3,8 +3,8 @@ This file contains all the DOI-related functions.
"""
import requests
import regex
import tools
from . import regex
from . import tools
def extract_doi_links(urls):

0
opendetex → reference_fetcher/opendetex

0
regex.py → reference_fetcher/regex.py

0
tools.py → reference_fetcher/tools.py

Loading…
Cancel
Save