Workdocumentation 2021-03-15

Proceedings Title Parser import via Pipeline

Goal: Transfer of PTP json results as WikiSon input to Openresearch (copies).

Unix pipeline to transfer includes getting a json from the PTP and passing it into the python file Converter.py which in turn passes it into the wikirestore. <graphviz> digraph pipeline { ptitle [ label="Proceedings Title" [ ptitle -> ptp ptp -> converter converter -> wikirestore wikirestore -> openresearch } </grapvhiz>

Usage

Example:

• https://dblp.org/db/conf/icse/index.html
• https://dblp.org/db/conf/mobilesoft/index.html
• https://dblp.org/rec/conf/icse/2020mobilesoft.html?view=bibtex

Proceedings title:

MOBILESoft '20: IEEE/ACM 7th International Conference on Mobile Software Engineering and Systems, Seoul, Republic of Korea, July 13-15, 2020

Import script

#!/bin/bash

################################################################################
# Help                                                                         #
################################################################################
Help()
{
   # Display Help
   echo "Please use the following arguments"
   echo "Args 1(Mandatory): Link to JSON"
   echo "Args 2(Mandatory): Target Wiki"
   echo "Args 3(optional): -ui for UI interface"
   echo
}

if [ -z "$1" ]; then
    echo "No arguments given"
    Help
elif [ -z "$2" ]; then
    echo "No target wiki specified"
    Help
else
    curl -H 'Accept:application/json' "$1" -s | python ./wikibot/Converter.py -stdin -ro | wikirestore -t $2 -stdinp $3
fi
</source

*Running example:
<source lang='bash'>
./transfer.sh 'http://ptp.bitplan.com/parse?titles=MOBILESoft+%2720%3A+IEEE%2FACM+7th+International+Conference+on+Mobile+Software+Engineering+and+Systems%2C+Seoul%2C+Republic+of+Korea%2C+July+13-15%2C+2020%0D%0A&examples=example10&format=json' ormk

Result

• https://confident.dbis.rwth-aachen.de/ormk/index.php?title=MOBILESoft_2020

Converter.py

import argparse
import sys
import json
from pathlib import Path


def ensureDirectoryExists(directory):
    Path(directory).mkdir(parents=True, exist_ok=True)

def getHomePath(localPath):
    '''
    get the given home path
    '''
    homePath=str(Path.home() / localPath)
    ensureDirectoryExists(homePath)
    return homePath

def Parser_to_Wiki(filePath=None,stdin=False,restoreOut=False):
    dicttemp={'|Acronym=':'acronym','|Title=':'title', '|Series=':'series','|Type=':'eventType','|Start date=':'start_date','|End date=':'start_date','|Submission deadline=':'Submission_Deadline'
              ,'|Homepage=':'homepage','|City=':'city', '|Country=':'country' ,'|wikiCFP ID=':'wikiCFPId'}

    if stdin:
        dict_f = json.load(sys.stdin)
    else:
        print(filePath)
        f=open(filePath, 'r')
        dict_f = json.load(f)
    l = ''
    for i in dict_f['events']:
        l = '{{Event\n'
        for ev in dicttemp:
            l += ev + str(i.get(dicttemp.get(ev))) + '\n'
        l += '}}'
        l = l.replace("=None\n", '=\n')
        if restoreOut:
            wiki_file = open(getHomePath('wikibackup/ptp') + '/' + str(i.get('acronym')) + ".wiki", "w")
            wiki_file.write(l)
            wiki_file.close()
            print(getHomePath('wikibackup/ptp') + '/' + str(i.get('acronym')) + ".wiki")
        else:
            wiki_file = open(getHomePath('wikibackup/ptp')+'/'+str(i.get('acronym'))+".wiki", "w")
            wiki_file.write(l)
            wiki_file.close()

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('-p', dest="path", help='path of the files')
    parser.add_argument('-stdin', dest="pipein", action='store_true', help='Use the input from STD IN using pipes')
    parser.add_argument('-ro', dest="ro", action='store_true', help='Output to STDOUT for wikirestore by pipe')
    args = parser.parse_args()
    print(args.path)
    Parser_to_Wiki(args.path, args.pipein, restoreOut=args.ro)

The output: https://confident.dbis.rwth-aachen.de/ormk/index.php?title=MOBILESoft_2020