Python: скрипт сравнения файлов из SVN-репозитория

Автор: | 11/21/2014
 

PythonЕсть необходимость в сравнении несколько конфигурационных файлов из двух бранчей.

Скрипт загружает их из репозитория с помощью модуля pysvn, сохраняет под различными именами и с помощью модуля difflib выполняет сравнение.

Потом — создаётся репорт, в который включены различающиеся строки, и отправляется на почту.

Сам скрипт:

#!/usr/bin/env python

import sys
import os
import re
import difflib
import pysvn
import smtplib
import mimetypes
import email
import email.mime.application
import logging
import time

if len(sys.argv) < 2:
    print('Usage: %s [old brahcn number] [new branch number]nExample:n%s 4.8 4.12' % (__file__, __file__))
    sys.exit(1)

LOGPATH = '/var/log/configchecks.log'

BRANCH_NUM_OLD_CONF = sys.argv[1]
BRANCH_NUM_NEW_CONF = sys.argv[2]

CONFIGS = ('services.PROD.properties', 'services.properties', 'dao.properties', 'dao.PROD.properties')

URL = 'https://www.domain.com/svn/repos/%s.x/resources/%s'

REPORT = []

logging.basicConfig(format = '%(filename)s - %(levelname)-3s [%(asctime)s] %(message)s ', filename=LOGPATH, level=logging.INFO)

def svn_export(url_old, url_new, local_old, local_new, file):

    client = pysvn.Client()

    try:

        logging.info('Extracting file %s...' % file)

        client.export(url_old, local_old)
        client.export(url_new, local_new)

        for i in (local_old, local_new):

            if os.path.isfile(i):
                logging.info('File %s saved.' % i)
            else:
                logging.info("ERROR: can't find file %s." % i)

    except pysvn._pysvn.ClientError as e:
        logging.info('ERROR: %s' % e)
def diff_configs(local_old, local_new, file):

    logging.info('Checking lines...')

    local_old = open(local_old, 'r').readlines()
    local_new = open(local_new, 'r').readlines()

    d = difflib.Differ()
    diff = difflib.ndiff(local_old, local_new)

    num = 1

    for line in diff:

        if not re.match('^.{0,3}#', line):

            if re.search('^-', line):
                logging.debug('File %s in line %d: %s' % (file, num, line))
                REPORT.append('File %s in line %d: %s' % (file, num, line))
        num += 1

def runall(old_branch, new_branch):

    for file in CONFIGS:

        local_old = ('%s_%s' % (file, old_branch))
        url_old = (URL % (old_branch, file))

        local_new = ('%s_%s' % (file, new_branch))
        url_new = (URL % (new_branch, file))

        svn_export(url_old, url_new, local_old, local_new, file)
        diff_configs(local_old, local_new, file)

def mail(old_branch, new_branch, report, date):

    sender = 'user@domain.com'
    to = ['user1@domain.com', 'user2@domain.com', 'user3@domain.com']

    msg = email.mime.Multipart.MIMEMultipart()

    msg['Subject'] = ('Configs checks for branches %s and %s' % (old_branch, new_branch))
    msg['From'] = 'Configuration manager <user@domain.com>'
    msg['To'] = 'APP team <user1@domain.com>'

    body = email.mime.Text.MIMEText("""

    Report created %s.

    %s

    """ % (date, ('n'.join(report))))

    msg.attach(body)

    logging.info('Preparing email.')
    smtpconnect = smtplib.SMTP('smtp.domain.com:25')
#    smtpconnect.set_debuglevel(1)
    smtpconnect.sendmail(sender, to, msg.as_string())
    smtpconnect.quit()
    logging.info('Mail sent.')

if __name__ == "__main__":
    logging.info('Starting extract and compare files for branches %s and %s at %s.' % (BRANCH_NUM_OLD_CONF, BRANCH_NUM_NEW_CONF, (time.strftime('%Y-%m-%d-%H:%M'))))
    runall(BRANCH_NUM_OLD_CONF, BRANCH_NUM_NEW_CONF)
    mail(BRANCH_NUM_OLD_CONF, BRANCH_NUM_NEW_CONF, REPORT, (time.strftime('%Y-%m-%d-%H:%M')))

Запуск:

$ ./svn_1.py 4.8 14.12

Лог:

svn_1.py - INFO [2014-11-20 16:02:34,298] Starting extract and compare files for branches 4.8 and 14.12 at 2014-11-20-16:02.
svn_1.py - INFO [2014-11-20 16:02:34,301] Extracting file services.PROD.properties...
svn_1.py - INFO [2014-11-20 16:02:43,135] File services.PROD.properties_4.8 saved.
svn_1.py - INFO [2014-11-20 16:02:43,135] File services.PROD.properties_14.12 saved.
svn_1.py - INFO [2014-11-20 16:02:43,136] Checking lines...
svn_1.py - INFO [2014-11-20 16:02:43,141] Extracting file services.properties...
svn_1.py - INFO [2014-11-20 16:02:52,653] File services.properties_4.8 saved.
svn_1.py - INFO [2014-11-20 16:02:52,654] File services.properties_14.12 saved.
svn_1.py - INFO [2014-11-20 16:02:52,655] Checking lines...
svn_1.py - INFO [2014-11-20 16:02:52,665] Extracting file dao.properties...
svn_1.py - INFO [2014-11-20 16:03:03,625] File dao.properties_4.8 saved.
svn_1.py - INFO [2014-11-20 16:03:03,626] File dao.properties_14.12 saved.
svn_1.py - INFO [2014-11-20 16:03:03,626] Checking lines...
svn_1.py - INFO [2014-11-20 16:03:03,671] Extracting file dao.PROD.properties...
svn_1.py - INFO [2014-11-20 16:03:14,695] File dao.PROD.properties_4.8 saved.
svn_1.py - INFO [2014-11-20 16:03:14,695] File dao.PROD.properties_14.12 saved.
svn_1.py - INFO [2014-11-20 16:03:14,696] Checking lines...
svn_1.py - INFO [2014-11-20 16:03:14,702] Preparing email.
svn_1.py - INFO [2014-11-20 16:03:15,358] Mail sent.

И письмо:

    Report:

File services.PROD.properties in line 29: —

File services.PROD.properties in line 30: — Url=https://domain.com/report

File services.properties in line 44: — Url=http://domain.com:11704/jj.dll

File dao.properties in line 66: —

File dao.properties in line 69: — jdbc.driverClassName=oracle.jdbc.driver.OracleDriver

File dao.properties in line 70: —

File dao.properties in line 73: — jdbc.url=jdbc:oracle:thin:@skynet.domain.com:1521:hh

File dao.properties in line 75: — jdbc.rf.username=user