Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dates to README.md table of versions #10

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Benjamin-Loison
Copy link

@Benjamin-Loison Benjamin-Loison commented Apr 21, 2024

The dates are guessed from the dates of the posts linked in Version column.

The following commands can be used to ease reviewing this pull request:

sed -i 's/None | //g' README.md
sed -i 's/[0-9][0-9]\/[0-9][0-9]\/[0-9][0-9] | //' README.md

Note that I manually verified and corrected the chronological order and not repeated dates.

As a starting point I used the following algorithm:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from lxml import html
import re
from datetime import datetime
import locale

# Necessary for parsing French dates.
locale.setlocale(locale.LC_TIME, 'fr_FR.utf8')

FILE_NAME = 'README.md'

with open(FILE_NAME) as f:
    lines = f.read().splitlines()

options = Options()
options.add_argument('-headless')
options.set_preference('permissions.default.image', 2)
browser = webdriver.Firefox(options=options)

#lines = lines[175:]

for lineIndex, line in enumerate(lines):
    #print(line)
    if line.startswith('| Version |'):
        line = f'| Date {line}'
    elif line.startswith('| - |'):
        line = f'| - {line}'
    elif line.startswith('| '):
        if 'Lounek' in line:
            continue
        match = re.search('\((http.*?)\)', line)
        print(lineIndex, line)
        tableDateStr = None
        try:
            url = match.groups()[0]
            print(url)
            if not 'http://www.dofus.com/?page=news&rubrique=v' in url:
                if not url.startswith('https://github.com/dofera/dofedex/releases/tag/'):
                    if match.span()[0] < line.index('|', 1):
                        entry = re.search('entry\d+', url)
                        #url = 'https://www.dofus.com/fr/forum/1751-dofus-retro/2396381-dofus-retro-mise-jour-1-41-reunification-dofus?page=1#entry13248306'

                        browser.get(url)

                        text = browser.page_source

                        tree = html.fromstring(text)

                        if entry is not None:
                            span = tree.xpath(f'//div[@id="{entry.group()}"]/div[2]/div[1]/div[1]/span[3]')
                            if span != []:
                                dateStr = span[0].attrib['title']
                                date = datetime.strptime(dateStr, '%d/%m/%Y - %H:%M:%S')
                            else:
                                # Should investigate this case.
                                date = None
                        else:
                            texts = list(tree.xpath('//span[@class="ak-subtitle"]')[0].itertext())
                            match len(texts):
                                case 2:
                                    dateStr = texts[1].lstrip()
                                    date = datetime.strptime(dateStr, '- %d/%m/%Y - %Hh%M')
                                case 3:
                                    dateStr = texts[2].strip()
                                    date = datetime.strptime(dateStr, '%d %B %Y - %H:%M:%S')
                                case 13:
                                    dateStr = texts[6].strip()
                                    date = datetime.strptime(dateStr, '%d %B %Y - %H:%M:%S')
                                case 14:
                                    dateStr = texts[7].strip()
                                    date = datetime.strptime(dateStr, '%d %B %Y - %H:%M:%S')
                    else:
                        # Should investigate this case.
                        date = None
                else:
                    date = None
            else:
                #dateStr = re.search('\((.*)\)', tree.xpath('//div[@class="column"]/p')[0].text_content()).groups()[0]
                dateStr = re.search('(\d{2}/\d{2}/\d{2})', tree.xpath('//title')[0].text_content()).groups()[0]
                date = datetime.strptime(dateStr, '%d/%m/%y')
            print(date)
            if date is not None:
                tableDateStr = date.strftime('%d/%m/%y')
        except:
            pass
        line = f'| {tableDateStr} {line}'
    lines[lineIndex] = line
    #print(line)

with open(FILE_NAME, 'w') as f:
    f.write('\n'.join(lines))

browser.quit()

There are 2 incorrect chronological ordered dates:

| 03/10/06 | 1.15.2 |
| 02/02/22 | 1.37.10 |

as other same day dates it is due to incorrect Version links.

Related to Benjamin_Loison/dofedex/issues/1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant