Skip to end of metadata
Go to start of metadata

When using external build nodes or CI labs, the logs still need to be push to a public, centralized server. The following are the steps needed to get access to push logs to nexus.akraino.org.

Step-by-step guide

  1. Create a helpdesk ticket (currently, go to Jira Support Desk; always-up-to-date list here: https://docs.releng.linuxfoundation.org/en/latest/helpdesk.html). The request should indicate that you are requesting "external log pushing permissions". You must provide the following information:
    1. A name for the log directory/silo (we have generally just been using company name, e.g. "att" or "ericsson").
    2. An LFID to be granted appropriate permissions on the Nexus server. Please create an ID specifically for this purpose at identity.linuxfoundation.org, rather than using a personal LFID.
  2. Pushing the logs from within your internal jobs can be handled however you like, but below is a suggested script (adapted from one provided by Andrew Wilkinson). Please note that we run a cleanup script that check the age of directories at a depth of 4 within the logs, so it is important to use the path of "${SILO}/job/${JOB_NAME}/${BUILD_NUMBER}".

    push_logs.sh
    # Deploying logs to LF Nexus log server ##
    # BUILD_NUMBER and JOB_NAME should be set by Jenkins
    
    NEXUS_URL=https://nexus.akraino.org
    SILO=<silo name provided above>
    JENKINS_HOSTNAME=<hostname/IP of your Jenkins host>
    BUILD_URL="${JENKINS_HOSTNAME}/job/${JOB_NAME}/${BUILD_NUMBER}/"
    NEXUS_PATH="${SILO}/job/${JOB_NAME}/${BUILD_NUMBER}"
    
    lftools deploy logs $NEXUS_URL $NEXUS_PATH $BUILD_URL
    
    echo "Logs uploaded to $NEXUS_URL/content/sites/logs/$NEXUS_PATH"

    Another user-suggested way to structure this script is to push all the log files within a given folder to Nexus, with the below suggested script (please note that this version has not been tested by the LF):

push_archives.sh
# Archive files to LF Nexus log server ##

NEXUS_URL=https://nexus.akraino.org
SILO=<silo name provided above>
NEXUS_PATH=<the destination folder on Nexus, such as $SILO/job>
WORKSPACE=<archives directory provided by the user, with the -p option below it specifies the file pattern as .log files>

lftools deploy archives -p '**/*.log' $NEXUS_URL $NEXUS_PATH $WORKSPACE
 
echo "Logs uploaded to $NEXUS_URL/content/sites/logs/$NEXUS_PATH"


This method requires installing lftools via pip: https://pypi.org/project/lftools/. The credentials for your service account's LFID should be stored in $HOME/.netrc 




4 Comments

  1. Hi Andrew Wilkinson,

    what is content inside .netrc file? What are needed parameters?

    machine nexus.akraino.org
    login <it this LFID or mail address>
    password

    Br,
    Mika Rautakumpu

  2. would suggest installing Nginx prior to upload the log.    



  3. I would rather suggest to use the S3 protocol without any third-party tools (lftools).

    It's supported by LFN common test frameworks such as Xtesting without any effort (3 env variables).

    It's already in place in OPNFV (centralized artifact repository - artifacts.opnfv.org) and CNTT RC cookbook (local artifact repository - Minio)

    CC: Tapio Tallgren Juha Kosonen