Contents

Info

Process, Project review and recommend sub-committee: Finalize request Templates for Blueprint and feature project, document to allow developers to get started. Develop and evolve the process by which blueprint and feature project proposals are reviewed, and ultimately approved with recommendation required documentation.

Sub-Committee Chair - Biswajit De.  Ele01 Dec 2020 /Dec 2021

Template 1 - Use case template

Template 2 - Blueprint family template

Template 3 - Blueprint species template

Membership

Please join the Process Sub-Committee mail list by self-adding within the Akraino Mail List Sub-Groups page. 

Note: Please ensure that both the name and email address for each member is listed on each sub-committee membership wiki page in order to properly set up CIVS voting when required.

Interested parties sign up:

Process, Project review and recommend, documentation sub-committee members(From Nov 2023~): 

We have started the process to elect a new chair for the Process, Project review and recommend, documentation sub-committee. 

The process started on 23 Nov and will continue until Noon 15 Dec 2023 (Pacific)  

The election process is two steps.  The first is for people to self-nominate for the position.  To do this, put a Y in the correct column if you wish to run for Chair.  The second step is to have an election for the position.  If there is only one person who has self nominated, then that person will be chair.  

Note: Please ensure that both the name and email address for each member is listed on each sub-committee membership wiki page in order to properly set up CIVS voting when required. 

Interested parties sign up:

Process, Project review and recommend, documentation sub-committee members(From Oct 2020~Nov 2023): 

Election - Oct 2020

Chair are supposed to be elected once a year and we are over due for an election.  Akraino runs off a self-nomination model, meaning that if you would like to run for Chair, please place a Y in the column.  If you do not want to run for the chair position, please put a N in the column.

The self-nomination is now open and will close at noon (Pacific Time) on 28 Oct.  If there are two or more nominees, we will proceed to an election.  If not, the self-nominated person will become Chair.

Completed Project Graduation Reviews

Graduation reviews

Meeting Content (minutes / recording / slides / other):

Please note minutes of meetings held for graduation reviews are not recorded below but rather in the Graduation reviews section of the sub committee wiki.

July 23rd, 2019

Andrew, Tina, Sukhdev, Tapio, Sharad

Family level PTLs

LF requires a single TSC named maintainer of all repos supported by it. If a Family of BPs utilizes common Family level code and as such Family level repos hierarchically above the individual BP species repos are required, LF requires a named person to be appointed as their maintainer before creation,

To resolve this is the P-SC considered 3 options:

(1) create a new PTL at the Family

(2) create a Akraino Coordination Role to be the authorized maintainer of Family level repos  (Akraino Technical Community Document#4.3.3Coordinators)

(3) ask LF to modify their processes to allow any of the BP PTLs to take the role of maintainer

The P-SC recommendation is to, when necessary, appoint a Family level Coordinator Role based on the recommendation of the Family's individual BPs' PTLs. This person could be one of the PTLs or another Akraino community member as proposed by the Family BP PTLs.

When approved the TSC shall then notify the LF once of the identity of appointee of each Family Coordinator Role (if they exist) and at the Family level in the Akraino wiki record the approved Family level (repo) coordinator.

Incase of any need or desire for change the Family BPs' PTLs shall make a new nomination for the coordinator role for the TSC to approve (who shall then update LF and the wiki).

Voter and TSC candidacy eligibility

The P-SC discussed the current contribution based approach to election voter eligibility and TSC candidacy. We noted that the four explicitly identified contribution types in the Technical Community Document ("code, code reviews, Wiki and documentation contributions,  Jira activities") are not exclusive and "other artifacts" can be considered by the TSC in determining voter and candidate eligibility.

However the P-SC recommends that

(1) The TSC includes the following as valid criteria for both assessments for the August 2019 election:

The PTLs of BP that have been included in the previous Akraino release (either as self certified or Mature or Core),

Sub Committee Chairs

 TSC Chairs

(2) Post the 2019 TSC elections that the Technical Community Document be revisited to explicitly include these as voter and candidacy qualification and also consider any other long term contributions the new TSC may consider likewise.

April 25th, 2019

Andrew, Sukhdev, Thor Chin

P-SC call at 4PM PT to review API GW proposal from Inswinstack

Agreed to introduce as a feature within a blueprint to allow both standalone deployment and wider deployment into other blueprints.

Both members on the process committee had seen and reviewed the material.

Recommendation to TSC is to approve with the note only one committer has currently been identified.

Feb 26, 2019

• attendees
  • Jim, Tina, Tapio, Andrew, Bill
• we reviewed the proposed wording for the use of the name “Akraino” in project titles
  • we decided we’ll review this internally…
  • “the use of the term Akraino certified BP or FP is restricted to BP or FP that have been verified by the TSC as reaching the minimum level of conformance to the Mature or Core levels”
• release 1 milestones
  • we’ll add this item to our docket
• we started discussing it wrt our Blueprint state table
  • how do the two processes fit together?
  • RC0, RC1, RC2
    • why 3?
    • Tapio related his experience from OPNFV – need some kind of tracking for sure, but don’t want it too detailed
• ideally the milestones would be real tasks, and demonstrable
• we brainstormed a list of milestone tasks that we might want to propose
  • first deployment in validation lab
  • first testing logs shipped up to CI/CD environment
  • peer Jenkins is set up & working
  • blueprint-specific test plan shared with the Community
  • lab is set up for CD
• we also discussed that the blueprint should iterate through M3-RC1 as many times as the Blueprint owner thinks makes sense

Feb 19, 2019

Akraino Naming/Badge

• Jenny raised the question – is the inclusion/exclusion of the word “Akraino” in the BP/FP name meaningful?

• should all names have Akraino in them, or should this only be added when the BP reaches Core?
• we agreed to consider the question between now & the next meeting, and to vote on the specific issue of the Akraino name/badge in the next Process sub-committee meeting

Further Review of TCD Section 3.3.4

• we picked up the discussion that we started last week - managed to make some progress on finalizing the updates to the first two states

Feb 12, 2019

Recording

Recording / Chat

Review of TCD Section 3.3.4

We discussed the following table, which expands on what’s already there in section 3.3.4, and adds…

Release Quality

• as discussed in previous meeting – just moved it to its own column

Release Number

• as discussed in previous meeting – an initial stab at it from last week's discussion

Deliverables / Exit Criteria

• pulled in the “review metrics” from section 3.3.7.1 – 4 

The intent is to simplify this section, putting everything in one place.  We got through the first two states this week... 

Feb 5, 2019

Attendees

• Jim Einarsson, Jenny Koerv, Aaron Byrd, Andrew Wilkinson, Frank Zdarsky, Mike Hunter, Bill Zvonar

Release Cadence

• we talked about proposing a cadence to the TSC
• currently, there’s talk of a second release in 6 months, but not of a cadence, per se
• we agreed to park this for later 

Proposed Changes from Last Process Sub-Committee Meeting

• we agreed to get those to vote at the TSC
• ACTION: Bill to get those on the agenda for the next TSC meeting

More Proposed Changes

• Andrew asked about MVP re: Incubation - MVP as an "outcome" - doesn't seem right 
• instead, it should say something like During incubation, an MVP-quality product will be demonstrated (Alpha).
• discussion on Alpha/Beta/GA vs. Incubation/Mature/Core ensued 
• we agreed on the following revised wording in section 3.3.4 (Project Lifecycle States and Reviews) of the TCD… Incubation
  • Project has resources, but is recognized to be in the early stages of development.  In the Incubation state, the goal is to progress the project from Alpha (MVP) quality to Beta quality.  The outcome is a minimum viable product (MVP) that demonstrates the value of the project and is a useful vehicle for collecting feedback, but is not expected to be used in production environments.
• Mature
  • Project is fully functioning and stable, has achieved successful releases.  In the Mature state, the goal is to progress the project from Beta quality to GA quality.  
    
  Core
  • Project provides value to and receives interest from a broad audience.  In the Core state, the project is at GA quality – additional functionality may be added in subsequent releases.
    
  Archived
  
  • Project can reach Archived state for multiple reasons.  Either project has successfully been completed and its artifacts provide business values, or project has been cancelled for unforeseen reasons (no value anymore, technical, etc.).  Project in any state can be Archived through a Termination Review.  In the Archived state, the project is Deprecated - no further bug fixes, security updates, etc. 
• ACTION: Bill to get this on the agenda for the next TSC meeting as well 

Checklists for Graduation 

• further discussion on clarifying/simplifying the deliverables that should be delivered in each state, and the exit criteria for graduating from one state to another
• stuff like security checklists, and other things that might be specific to a given BP Family
• also should add language around release numbering - e.g. is "Mature" always Release 1? or Core? 
• we agreed to start a table of such details
• ACTION: Bill to start the table of States, Deliverables, etc.

Jan 29, 2019

Attendees: Jim Einarsson, Tina Tsou, Tapio Tallgren, Andrew Wilkinson, Bill Zvonar

Trouble Starting the Meeting

• likely because Zoom bridge wasn’t completely closed at the end of the TSC Working Group meeting – note to selves for our meeting

Review Feature Project Template (per last TSC)

• Context
• Wiki: https://wiki.akraino.org/display/AK/Akraino+Blueprint+Validation+Framework
• the Process Sub-Committee was asked in the last TSC to review this feature project
• Jim noted that Jenny had questioned in the last TSC meeting whether we (the Process Sub-Committee) should “approve” this project into Incubation – or rather, should we limit ourselves to saying whether the template is ok or not
• Jim further noted that we had created some “Blueprint” criteria, but not for a “Feature Project” – so would it be the same?
• Committers
• Tapio asked if there should be Committers identified at this point – this led into a discussion about *when* in the project lifecycle the Committers need to be identified

TCD Changes re: PTL/Committers & Project “Start”

• would the project just die if there were no Committers? or would it just go On Hold?
• can you enter Incubation *then* identify Committers?
• you have to have at least 1 Committer *Company* identified
• Committers to be named up until the PTL election
• ACTION: two changes proposed to the TCD – bring to the TSC meetingindicate when Committers are to be added to a Feature Project during PTL elections, after which the project ‘starts’ 
  • section 3.2.2.1
  • Initial Committers for a project will may be specified at project creation and up until PTL election.  The project is not started until after the PTL election.indicate that a Project “starts” at the beginning of the Incubation state
    • section 3.3.4in the “Incubation” row of the table in section 3.3.4
      • Project has resources, but is recognized to be in the early stages of development. The project is considered to have started upon successful PTL election.  The outcome is a minimum viable product (MVP) that demonstrates the value of the project and is a useful vehicle for collecting feedback, but is not expected to be used in production environments.
• in terms of the content of the Feature Project Template, we questioned what is meant by ‘Akraino ready / validated’?

didn’t get to these agenda items, will table them for next week’s meeting

• Discuss/align on bronze/silver/gold
• Kick off splash page /spreadsheet for “mature” graduation
• Other topics

Jan 15, 2019

Attendees: Tina Tsou, Jenny Koerv

Formal motion at next TSC meeting to vote on action to revise

1. TCD sections 3.3.7.1 and 3.3.4 discussed in 11/30 Process Sub-Committee and shared/agreed/implemented at TSC F2F 12/6-7 (summary/language recommendation here: Remove This Page 1). 
2. Clean up/changes to TCD 3.3.8.1 and 3.3.2.3.1 language discussed/presented in TSC Meeting 1/10 (see minutes/slide 4 here Technical Steering Committee (TSC)).
3. More specific language in Maturity Review 3.3.7.2.
4. Successful participation in at least two releases: The project demonstrates stable output (code base, documents) within its history of releases in accordance with the release policy.
5. Architecture has been reviewed by the Architecture Committee CI/CD Sub-Committee, TSC and presented to broader Akraino community?
6. Project Contributors have provided a validation lab with exact configuration required by the project to connect with Akraino CI and demonstrate CD.  The environment should be reviewed and endorsed by the CI/CD Sub-Committee.
   
7. Project is active and contributes to Akraino: The project demonstrates a stable or an increasing number of commits and/or contributions across recent releases measurable by TSC discretion. Contributions are commits which got that have been merged to a repository of an Akraino project repository or a related upstream project. Commit examples are can for example be patches to update the requirements document of a project, code addition to an Akraino or upstream project repository, new test cases and so forth.
   
8. Mature artifacts produced: The project demonstrates that the artifacts produced by the project are deployable (where applicable) and have been successfully deployed, configured and used by at least two independent end users (typically, service providers) who have publicly documented their support on the Akraino wiki and within the accompanying project documentation.
   

Ask for TSC to review the following as considerations for Maturity Review criteria:

1. LF CII best practices: https://bestpractices.coreinfrastructure.org/en
2. Contributor Code of Conduct: https://www.contributor-covenant.org/version/1/2/0/code-of-conduct.html

The advantage and desire to have a simple graduation splash page on the Akraino Wiki (Under Process Sub-Committee or otherwise).  That simply outlines the Project LifeCycle and Graduation criteria (comparable to the CNCF page here: https://github.com/cncf/toc/blob/master/process/graduation_criteria.adoc ).  Jenny to take a stab at it…

The need for the TSC to agree on detail language of Review votes for graduation (Majority?  2/3?).  The follow up need to document that the TSC vote is to determine whether Criteria have been adequately met for advancement. Note: Jenny afterthought – did not formally get to this in this meeting.

Should section 3.3.7.3 Core Review include the Conformance language?  Should Conformance badge warrant the “Akraino” brand and define the “Akraino” value? Jenny afterthought – sent related note to TSC in context of “Akraino Portal Feature Project”

Jan 8, 2019

• Update Technical Community Document criteria
• To achieve maturity phase, at least two independent users should demonstrate/document deployment subject to TSC review, must have achieved multiple releases and have a release plan, must be able to demonstrate/document increasing contributions.
• Minor releases – i.e point releases (in maturity phase) and major release (to achieve core phase) of a blueprint should be delivered
• Conformance criteria will be defined by the TSC for each blueprint and feature project would be established via the Akraino Conformance Validation feature project and the results of conformance testing for each blueprint / feature project would be used in TSC voting.  Feature project criteria should include validation of upstream community contribution to the feature project to avoid duplication of effort.
• For feature projects to move into maturity phase, the submitted would be required to identify the blueprint(s) that have benefitted from the feature project
• Section 3.3.8.1 of the Technical Community Document defines a 'Release Plan' for Maturity.  List is redundant/repeated.  Need to edit TCD. 
• 3.3.2.3.1 language calls out Feature project testing in Community CI lab, and integration of feature projects into a BP in Community CI lab. Might be problematic if CI lab is not mirror of Flock lab testing.  Need to change the language in the TCD to say that the integration / feature projects need to be deployed to the Akraino Validation (Flock) Lab.
• For code submission criteria, LF ID (approved project committer – PTL manages the process), Akraino CI/CD process, specifications defined and documented, approvers of specifications.

November 30, 2018  minutes

30 Nov 2018

Attendees:

Jim Einarsson, Windriver

Andrew Wilkinson, Ericsson

Jenny Koerv, Intel

Frank Zdarsky, Red Hat

Adnan Saleem, Radysis

The Process Sub-Committee is generally agreed that further discussion on BP submission, review and graduation methodology, clarification on Technical Doc language and better understanding of CI/CD environment and Flock lab requirements should be further discussed before scheduled BP reviews 12/6.  This is mainly for two broad reasons

1. Further understanding of the CI/CD environment is critical to submit and review a blueprint as defined by the Technical Community Document (TCD)
2. The TCD itself is confusing, redundant and insufficient as written to efficiently conduct and conclude a fair BP or project review process.
   
   

There are currently three places in the technical document where criteria and/or methodology relevant to the F2F BP review process are defined:

3.3.2.2 Akraino Edge Stack Integration Projects (Blueprints)

3.3.4  Project Lifecycle States and Reviews

3.3.7.1 Incubation Review

The Process Sub-Committee would like to propose revising the following language in the TCD 3.3.2.2 Blueprint section (which should not require such detail for the Incubation Review state outlined in 3.3.4—and is currently impossible to speak to at this review cycle without an agreed and operating CI/CD foundation):

Current TCD language (strikes represent the language we would like revised):

The requestor should demonstrate the following aspects to the TSC:

1. Each initial blueprint is encouraged to take on at least two Committers from different companies
2. Complete all templates outlined in this document
3. 3.       A lab with exact configuration required by the blueprint to connect with Akraino CI and demonstrate CD. User should demonstrate either an existing lab or the funding and commitment to build the needed configuration.
4. 4.       Blueprint is aligned with the Akriano Edge Stack Charter
5. Blueprint code that will be developed and used with Akraino repository should use only Open Source software components either from upstream or Akriano projects.
6. For new blueprints submission, the submitter should review existing blueprints and ensure it is not a duplicate blueprint and explain how the submission differs. The functional fit of an existing blueprint for a use case does not prevent an additional blueprint being submitted.

Process Sub-Committee proposed language:

The requestor should demonstrate the following aspects to the TSC:

1. Each initial blueprint is encouraged to take on at least two Committers from different companies
2. Complete all templates outlined in this document
3. Prepared to commit lab resources to support collaborative development and validation testing
4. Blueprint is aligned with the Akraino Edge Stack Charter and Technical Community Document
5. Blueprint code that will be developed and used with Akraino repository should use only Open Source software components either from upstream or Akriano projects.
6. For new blueprints submission, the submitter should review existing blueprints and ensure it is not a duplicate blueprint and explain how the submission differs. The functional fit of an existing blueprint for a use case does not prevent an additional blueprint being submitted

The Process Sub-Committee are agreed on the minimum criteria defined by the Proposal -> Incubation “State” (Section 3.3.4), and intend to use this language as our guide for the BP Incubation Reviews, and Incubation graduation recommendation:

Proposal State Criteria:

Project doesn’t really exist yet

May not have real resources

Is proposed and is expected to be created due to business needs (documented) 

Incubation State Criteria (paraphrased with explanation):

-Project has resources (commitment needed for Incubation Review, specifics not necessary at this time)

-Project in early stages of development (commitment needed, too preliminary for HW detail, Committer bios, exact tooling w/o CI/CD environment foundation and clarity)   

-Outcome of Incubation state is MVP (goals are demonstrating project value and gathering feedback);

-Not for production deployment (understood and agreed for Incubation State deliverable)

The Process Sub-Committee are not comfortable with the current language in 3.7.7.1 and find it too progressive and inconsistent with the above Incubation State description and targeted outcome.  We would like to propose revising the criteria identified under the 3.7.7.1 section of the TCD as follows:

Current TCD language (strikes represent the language we would like revised):

 Artifacts expected for Incubation Review:

Name of the project is appropriate (no trademark issues etc.);

Proposed repository name is all lower-case without any special characters

Project contact name, company and email are defined and documented
Description of the project goal and its purpose are defined
Scope and project plan are well defined
Resources committed and available 
Contributors identified
Initial list of committers identified (elected/proposed by initial contributors)  -
Meets Akraino TSC Policies
Proposal has been socialized with potentially interested or affected projects and/or parties
Cross Project Dependencies (XPDs). In the case where a project will require changes in other projects, those projects are listed in the proposal, and a sponsoring developer in the project has been identified
Tools have been identified and discussed with relevant partners (Linux Foundation, IT).

Once the project passes the review, the tools chain must be created within one week. Tools encompass Configuration Management, CI/CD, Code Review, Testing, Team Wiki, End Users documentation (not exhaustive)

Process Sub-Committee proposed language:

 Artifacts expected for Incubation Review:

Name of the project is appropriate (no trademark issues etc.)

Proposed repository name is all lower-case without any special characters
Project contact name, company and email are defined and documented
Description of the project goal and its purpose are defined
Scope and project plan are well defined
Prepared to commit resources to each proposed blueprint species.
Contributors identified
Meets Akraino TSC Policies

Cross Project Dependencies (XPD) identified with upstream.

Other notes and recommendations:

1. CI/CD discussion slated for Day 2 of Akraino TSC F2F next week should happen Day 1, and prior to BP reviews .
1. Primary concerns are around not knowing what the CI environment will look like to plan tooling
2. The CD environment being closed/behind firewall make it hard to collaboratively develop, test and troubleshoot for the agreed Contributors and community.
2. We are not sure that a TSC vote should determine BP or project graduation from Phase to Phase to GA to EOL, etc.  Would like the TSC to consider whether meeting a certain set of validated criteria are met (Conformance model?) should be voted on rather than the TSC voting on moving a particular BP forward itself.
3. The Project and Project lifecycle (~3.4) sections of the TCD should probably precede the BP section (3.3.2.2) since a BP is an Integration Project.
4. We need an agreed/documented way to prioritize the BPs for review/evaluation (Date?  Maturity? # Contibutors?)
5. We need an agreed/documented way to provide feedback to BP or project submitters after Reviews and an agreed and allotted amount of time to give Submitters to address gaps for a follow on submission.  Not really fair to vote on these right after review and not give the Submitters/Contributors time to correct/resubmit.
6. Need clearer language in TCD around what “Launch” actually means and where to apply it within the document/State process.
7. Need more refined language/plan from Upstream Sub-Committee around process/interface for project dependency socialization and integration.
8. Need pointer to Akraino branded templates (slides?) from Jackie/LF.

AR – Jenny to provide spreadsheet Incubation Review cheat sheet to Process Sub-Committee for feedback/revision

AR – Jenny to provide first pass notes

AR- Frank, Andrew, Adnan to add anything I missed/got wrong to notes

AR—Jim to socialize/represent and summarize concerns/recommendations to broader TSC and TSC Chairs

December 6, 2018 TSC Meeting Blueprint Review:

-          Attendees

o   Jim Einarsson, Jenny Koerv, Aaron Byrd, Andrew Wilkinson, Frank Zdarsky, Mike Hunter, Bill Zvonar

-          Release Cadence

o   we talked about proposing a cadence to the TSC

o   currently, there’s talk of a second release in 6 months, but not of a cadence, per se

o   we agreed to park this for later

-          Proposed Changes from Last Process Sub-Committee Meeting

o   we agreed to get those to vote at the TSC

o   ACTION: Bill to get those on the agenda for the next TSC meeting

-          More Proposed Changes

o   Andrew asked about MVP re: Incubation – MVP as an “outcome” – doesn’t seem right

o   instead, it should say something like During incubation, an MVP-quality product will be demonstrated (Alpha).

o   discussion on Alpha/Beta/GA vs. Incubation/Mature/Core ensued

o   we agreed on the following revised wording in section 3.3.4 (Project Lifecycle States and Reviews) of the TCD…

o   ACTION: Bill to get this on the agenda for the next TSC meeting as well

-          Checklists for Graduation

o   further discussion on clarifying/simplifying the deliverables that should be delivered in each state, and the exit criteria for graduating from one state to another

o   stuff like security checklists, and other things that might be specific to a given BP Family

o   also should add language around release numbering – e.g. is “Mature” always Release 1? or Core?

o   we agreed to start a table of such details

o   ACTION: Bill to start the table of States, Deliverables, etc.