...
- InferenceModel
- TrainingModel
Also the whole system will run in a hybrid mode. By hybrid, we mean KubeEdge will manage both cloud worker nodes and edge nodes. For the cloud running components, we can leverage the full power of standard K8s, i.e. pods, deployments, services, ingress/egress, loadbanancer etc. This means cloud components will be deployed in a way exactly the same as done in a standard Kubernetes cluster. On the other hand, edge components will leverage the KubeEdge framework.
InferenceModel CRD
| Code Block | ||||
|---|---|---|---|---|
| ||||
// InferenceModelSpec defines the desired state of an InferenceModel.
// Two ways of deployment are provided. The first is through a docker image.
// The second is through a data store, with a manifest of model files.
// If image is provided, manifest and targetVersion will be ignored.
type InferenceModelSpec struct {
ModelName string `json:"modelName"`
DeployToLayer string `json:"deployToLayer"`
FrameworkType string `json:"fraemworkType,omitempty"`
NodeSelector map[string]string `json:"nodeSelector,omitempty"`
Image string `json:"image,omitempty"`
Manifest []InferenceModelFile `json:"manifest,omitempty"`
TargetVersion string `json:"targetVersion,omitempty"`
Replicas *int32 `json:"replicas,omitempty"`
}
// InferenceModelFile defines an archive file for a single version of the model
type InferenceModelFile struct {
Version string `json:"version,omitempty"`
DownloadURL string `json:"downloadURL,omitempty"`
Sha256sum string `json:"sha256sum,omitempty"`
}
// InferenceModelStatus defines the observed state of InferenceModel
type InferenceModelStatus struct {
URL string `json:"url,omitempty"`
} |
An instance of the InferenceModel specifies a single serving service for the provided model. We
Two ways of deployment
The InferenceModel CRD supports both ways of deployment. If image is provided, manifest and targetVersion will be ignored.
| Deployment method | Pros | Cons |
|---|---|---|
| Docker image | ||
| Machine learning model file manifests |