Seldon Core Installation on Kubernetes (OpenShift)

AI & Data Engineering
6 min readJun 30, 2021


In this post, we will explore the installation of Seldon Core (the latest stable) with OpenShift Kubernetes.

Seldon Core is an open-source platform for deploying machine learning models on a Kubernetes cluster.


  • Kubernetes cluster ≥ 1.12. For Openshift it requires version 4.2 or higher
  • Installer method: Helm ≥ 3.0
  • Ingress: Istio & Ambassador

Let's first understand in brief, what is HELM and Ingress like Istio, and Ambassador.


In short, HELM is a package manager for Kubernetes. It is just like “yum” or “Maven” or “apt”. You can find packages in ArtifactHub


Ingress is the incoming traffic from the outside world into the cluster.

Ambassador is a Kubernetes ingress controller/API gateway that uses Envoy. And Istio is a service mesh that happens to have its own ingress mechanism. Istio also uses Envoy as sidecar proxies. We will discuss in a separate post about service mesh and sidecar patterns.

Installation Steps

Step1: Install HELM

curl -fsSL -o
chmod 700

Step2: Install Istio

  1. Download the Istio release
curl -L | sh -

2. Change the directory and get into the installed root directory of Istio

cd istio-1.10.2
export PATH=$PWD/bin:$PATH

3. By default, OpenShift doesn’t allow containers running with user ID 0. You must enable containers running with UID 0 for Istio’s service accounts by running the command below. Here, we are assuming that we will maintain the default Istio namespace: “istio-system”

$ oc adm policy add-scc-to-group anyuid system:serviceaccounts:istio-system

4. Install Istio. Here, use the profile as openshift

$ istioctl install --set profile=openshift

Step3: Install Seldon

  1. Create namespaces for Seldon
$ kubectl create namespace seldon
$ kubectl create namespace seldon-system

2. Apply the Security context constraints (scc) on both Seldon namespaces: seldon and seldon-system

$ oc adm policy add-scc-to-group anyuid system:serviceaccounts:seldon-system$ oc adm policy add-scc-to-group anyuid system:serviceaccounts:seldon

3. Update the kubeconfig for the seldon namespace

kubectl config set-context $(kubectl config current-context) --namespace=seldon

4. Install seldon-core using HELM. Here, I am enabling both Istio and Ambassador

helm install seldon-core seldon-core-operator \
--repo \
--set ambassador.enabled=true \
--set istio.enabled=true \
--set usageMetrics.enabled=true \
--namespace seldon-system

5. Check the seldon-core deployment status

kubectl get deployment -n seldon-system

6. We need an istio gateway installed in the istio-system namespace. By default, we assume one called seldon-gateway.

Let's create a YAML file with the name seldon-gateway.yaml and paste the below content as mentioned here:

kind: Gateway
name: seldon-gateway
namespace: istio-system
istio: ingressgateway # use istio default controller
- port:
number: 80
name: http
protocol: HTTP
- "*"

Apply the file for deployment using kubectl

kubectl create -f seldon-gateway.yaml

We can get the status of the deployment using kubectl rollout.

kubectl rollout status deploy/seldon-controller-manager -n seldon-system

Step4: Install Ambassador

  1. Add HELM repo for Ambassador

helm repo add datawire
helm repo update

helm repo add datawire
helm repo update

2. Installing ambassador

helm install ambassador datawire/ambassador \
--set \
--set enableAES=false \
--set crds.keep=false \
--namespace seldon

3. Check the deployment status

kubectl rollout status deployment.apps/ambassador -n seldon

4. Check the final status of deployment under seldon and seldon-system

kubectl get deployments -n seldon
kubectl get deployments -n seldon-system
kubectl get pods -n seldon-system
kubectl get pods -n seldon
kubectl get pods -n istio-system

5. The below command will forward traffic from the local 8003 port to port 8080 of the Ambassador ingress gateway pod within Kubernetes. Ambassador will then route this traffic to your Seldon deployments.

kubectl port-forward $(kubectl get pods -n seldon -l -o jsonpath='{.items[0]}') -n seldon 8003:8080

This listener will wait for ingress traffic to forward on from port 8003 to the pod port of 8080

Step5: ML Model Deployment

  1. Define the configuration of the SeldonDeployment, which will orchestrate the serving of your model. Here, we are taking an example of “Scikitlearn IRIS” ML model deployment.

The yaml file looks as below:

The deployment file will download sklearn-iris image for 0.1 version.The imagePullPolicy is “IfNotPresent” and name of the container is“sklearn-iris-classifier”. The endpoint is “REST”, type is MODEL and replica is 1.

kind: SeldonDeployment
name: seldon-deployment-example
name: sklearn-iris-deployment
- componentSpecs:
- spec:
- image: seldonio/sklearn-iris:0.1
imagePullPolicy: IfNotPresent
name: sklearn-iris-classifier
children: []
type: REST
name: sklearn-iris-classifier
type: MODEL
name: sklearn-iris-predictor
replicas: 1

Apply the yaml file for the deployment using kubectl create

kubectl create -f sklearn_iris_deployment.yaml

Check the deployment using “rollout” command. Please note that there is a seldon-xxx deployment and a pod seldon-xxx running under seldon namespace.

It shows, deployment successfully rolled out.

Step6: Test the Model deployment

You can test the deployment by throwing some features to the REST endpoint

The classifier expects an array of the form [Sepal Length, Sepal Width, Petal Length, Petal Width]. You can implement this simply using curl by running the following command:

curl -s http://localhost:8003/seldon/seldon/seldon-deployment-example/api/v0.1/predictions -H "Content-Type: application/json" -d '{"data":{"ndarray":[[5.964,4.006,2.081,1.031]]}}'

This should return a response that confirms the classification of your iris alongside confidence scoring

e.g. {"data":{"names":["t:0","t:1","t:2"],"ndarray":[[0.9548873249364169,0.04505474761561406,5.7927447968952436e-05]]},"meta":{}}

This example is suggesting that the iris is a Setosa with 95% confidence

So, that it!

In summary, we have seen

  • End-to-end deployment of Seldon Core
  • Deployment of Ingress like Istio and Ambassador
  • Deployment of ScikitLearn ML model in Seldon Core
  • Finally, testing the model



AI & Data Engineering

A Data Enthusiast, Lead Architect with 17 yrs experience in the field of AI Engineering, BI, Data Warehousing, Dimensional modeling, ML and Big Data