Versions Compared

Old Version 8

changes.mady.by.user zifan wu

Saved on

compared with

New Version 9

changes.mady.by.user zifan wu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

FATE-Flow REST API

  • HTTP Method: POST

  • Content-Type: application/json

DataAccess

/v1/data/upload

Introduction

FATE-Flow is the job scheduling system of the federated learning framework FATE, which realizes the complete management of the federated learning job life cycle, including data input, training job scheduling, indicator tracking, model center and other functions. It provides CLI and HTTP API for users to manage data and federated jobs.

FATE-Flow REST API

  • HTTP Method: POST

  • Content-Type: application/json

DataAccess

/v1/data/upload

  • request structure

    • namespace: Required,String: upload data table namespace

    • table_name: Required,String: upload data table name

    • work_mode: Required,

    request structure

    • namespace: Required,String: upload data table namespace

    • table_name: Required,String: upload data table name

    • work_mode: Required,Integer: eggroll’s working mode

    • file: Required, String: upload file location

    • head: Required,Integer: determine if there is a data header

    • partition: Required,Integer: set the number of partitions to save data

    • module: Optional,String: If you need to use the data of the machine where the FATE-Flow server is located, this value is not empty.

    • use_local_data: Optional,String: If you need to use the data of the machine where the FATE-Flow server is located, this value is 0.

    • drop: Optional, Integer: When the cluster deployment uses the same table to upload data, it is necessary to carry the drop parameter,0 represents overwriting upload, 1 represents deleting the previous data and re-uploading

  • response structure

    • job_id: upload job id,String

    • data: return data for submitting job ,Object

...

  • request structure

    • create: Optional, Boolean: whether to create

    • namespace: Optional,String: download data table namespace, need to be used with table_name

    • table_name: Optional,String: download data table name, need to be used with namespace

    • local: Optional,Object: local configuration

    • role: Optional,Object: role information

    • data_type: Optional,String: download file data type

    • gen_table_info: Optional,Boolean: tag table information

  • response structure

    • retcode: return code, Integer

    • retmsg: return code description, String

    • data: table information

/v1/table/delete

...

request structure

  • namespace: Optional,String: download data table namespace, need to be used with table_name

  • table_name: Optional,String: download data table name, need to be used with namespace

    • namespace

    • local: Optional,Object: local configuration

    • role: Optional,Object: role information

    • data_type: Optional,String: download file data type

    • gen_table_info: Optional,Boolean: tag table information

  • response structure

    • retcode: return code, Integer

    • retmsg: return code description, String

    • data: table information

/v1/table/delete

  • request structure

    • namespace: Optional,String: download data table namespace, need to be used with table_name

    • table_name: Optional,String: download data table name, need to be used with namespace

  • response structure

    • retcode: return code, Integer

    • retmsg: return code description, String

    • data: table information


FATE-Flow Client Command Line Interface

Usage

python fate_flow_client.py -f $command

JOB_OPERATE

submit_job

  • description: submit a pipeline job

  • parameter:

    • -c -config: runtime conf path, Required

    • -d -dsl: dsl path, Required

    python fate_flow_client.py -f submit_job -c examples/test_hetero_lr_job_conf.json -d examples/test_hetero_lr_job_dsl.json

stop_job

  • description: cancel job or stop job

  • parameter:

    • -j -job_id: job id, Required

    python fate_flow_client.py -f stop_job -j $job_id

query_job

  • description: query job information by filters

  • parameter:

    • -j -job_id: filter by job id, Optional

    • -r -role : filter by role, Optional

    • -p -party_id: filter by party id, Optional

    • -s -status: filter by status, Optional

    python fate_flow_client.py -f query_job -j $job_id

clean_job

  • description: clean processor,data table and metric data

  • parameter:

    • -j -job_id: filter by job id, Optional

    • -r -role : filter by role, Optional

    • -p -party_id: filter by party id, Optional

    • -cpn -component_name: component name, Optional

    python fate_flow_client.py -f clean_job -j $job_id

data_view_query

  • description: query data view information by filters

  • parameter:j -job_id: filter by job id, Optionalr -role : filter by role, Optionalp -party_id: filter by party id, Optionals -status: filter by status, Optional

python fate_flow_client.py -f data_view_query -j $job_id

JOB

job_config

  • description: download the configuration of this job

  • parameter:

    • -j -job_id: job id, Required

    • -r -role : role, Required

    • -p -party_id: party id, Required

    • -o -output_path: config output directory path, Required

    python fate_flow_client.py -f job_config -j $job_id -r $role -p $party_id -o $output_path

job_log

  • description: download the log of this job

  • parameter:

    • -j -job_id: job id, Required

    • -o -output_path: config output directory path, Required

    python fate_flow_client.py -f job_log -j $job_id -o $output_path

TASK_OPERATE

query_task

  • description: query task information by filters

  • parameter:

    • -j -job_id: filter by job id, Optional

    • -cpn -component_name: filter by component name, Optional

    • -r -role : filter by role, Optional

    • -p -party_id: filter by party id, Optional

    • -s -status: filter by status, Optional

    python fate_flow_client.py -f query_task -j $job_id

TRACKING

component_parameters

  • description: query the parameters of this component

  • parameter:

    • -j -job_id: job id, Required

    • -cpn -component_name: component name, Required

    • -r -role: role, Required

    • -p -party_id: party id, Required

    python fate_flow_client.py -f component_parameters -j $job_id -r $role -p $party_id -cpn $component_name

component_metric_all

  • description: query all metric data

  • parameter:

    • -j –job_id: job id, Required

    • -cpn –component_name: component name, Required

    • -r –role: role, Required

    • -p –party_id: party id, Required

    python fate_flow_client.py -f component_metric_all -j $job_id -r $role -p $party_id -cpn $component_name

component_metrics

  • description: query the list of metrics

  • parameter:

    • -j –job_id: job id, Required

    • -cpn –component_name: component name, Required

    • -r –role: role, Required

    • -p –party_id: party id, Required

    python fate_flow_client.py -f component_metrics -j $job_id -r $role -p $party_id -cpn $component_name

component_output_model

  • description: query this component model

  • parameter:

    • -j –job_id: job id, Required

    • -cpn –component_name: component name, Required

    • -r –role: role, Required

    • -p –party_id: party id, Required

    python fate_flow_client.py -f component_output_model -j $job_id -r $role -p $party_id -cpn $component_name

component_output_data

  • description: download the output data of this component

  • parameter:

    • -j –job_id: job id, Required

    • -cpn –component_name: component name, Required

    • -r –role: role, Required

    • -p –party_id: party id, Required

    • -o –output_path: config output path, Required

    • -limit –limit: Limit quantity, Optional

    python fate_flow_client.py -f component_output_data -j $job_id -r $role -p $party_id -cpn $component_name -o $output_path

component_output_data_table

  • description: view table name and namespace

  • parameter:

    • -j –ob_id: job id, Required

    • -cpn –component_name: component name, Required

    • -r –role: role, Required

    • -p –party_id: party id, Required

    python fate_flow_client.py -f component_output_data_table -j $job_id -r $role -p $party_id -cpn $component_name

DATA

download

  • description: download table

  • parameter:

    • -c –config: config path, Required

    python fate_flow_client.py -f download -c examples/download_guest.json

upload

  • description: upload table

  • parameter:

    • -c –config: config path, Required

    • -drop –drop: Operations before file upload, Optional

    python fate_flow_client.py -f upload -c examples/upload_guest.json
python fate_flow_client.py -f upload -c examples/upload_guest.json -drop 0(or1)

upload_history

  • description: upload table history

  • parameter:

    • -j –job_id: job id, Optional

    • -limit –limit: Limit quantity, Optional

    python fate_flow_client.py -f upload_history -j $job_id
python fate_flow_client.py -f upload_history -limit 5

Table

table_info

  • description: query table information

  • parameter:

    • -n –namespace: namespace, Required

    • -t –table_name: table name, Required

    python fate_flow_client.py -f table_info -n $namespace -t $table_name

table_delete

  • description: delete table

  • parameter:

    • -n –namespace: namespace, Optional

    • -t –table_name: table name, Optional

    • -j –job_id: job id, Optional

    • -cpn –component_name: component name, Optional

    • -r –role: role, Optional

    • -p –party_id: party id, Optional

    python fate_flow_client.py -f table_delete -n $namespace -t $table_name
python fate_flow_client.py -f table_delete -j $job_id

Model

load

  • description: load model.

  • parameter:

    • -c –config: config path, Required

    python fate_flow_client.py -f load -c $conf_path

  • response example:

    {
    "data": {
        "detail": {
            "guest": {
                "9999": {
                    "retcode": 0,
                    "retmsg": "xxx"
                }
            },
            "host": {
                "10000": {
                    "retcode": 0,
                    "retmsg": "xxx"
                }
            }
        },
        "guest": {
            "9999": 0
        },
        "host": {
            "10000": 0
        }
    },
    "jobId": "xxxxxxxxxxxxxxxx",
    "retcode": 0,
    "retmsg": "success"
}

bind

  • description: bind model.

  • parameter:

    • -c –config: config path, Required

    python fate_flow_client.py -f bind -c $conf_path

  • response example:

    {
    "retcode": 0,
    "retmsg": "service id is xxx"
}

store

  • description: store model

  • parameter:

    • -c –config: config path, Required

    python fate_flow_client.py -f store -c $conf_path

restore

  • description: restore mode

  • parameter:

    • -c –config: config path, Required

    python fate_flow_client.py -f restore -c $conf_path

export

  • description: export model

  • parameter:

    • -c –config: config path, Required

    python fate_flow_client.py -f export -c $conf_path

import

  • description: import model

  • parameter:

    • -c –config: config path, Required

    python fate_flow_client.py -f import -c $conf_path

response structure

...

retcode: return code, Integer

...

retmsg: return code description, String

...

Next  Previous