vdaas/vald-tensorflow-ingress-filter repository overview

⁠Vald Tensorflow Ingress Filter

vald-tensorflow-ingress-filter is one of the official ingress filter components provided by Vald.

Its custom logic requires the input of the Tensorflow SavedModel as a request and outputs the result from the Tensorflow SavedModel as the request of the Vald Agent.

Using this component lets users vectorize various data such as text and images using the Tensorflow SavedModel only inside the Vald cluster without external APIs.

⁠Requirement

• Libraries: kubectl, Helm, grpcio, numpy, tensorflow, vald-client-python
• Others: Vald cluster

⁠Get Started

vald-tensorflow-ingress-filter is used for ingress filter component of the Vald cluster, which means it should be used on the Kubernetes cluster, not the local environment or Docker.

The steps are following:

1. Deploy vald-tensorflow-ingress-filter

   git clone https://github.com/vdaas/vald-tensorflow-ingress-filter.git
   kubectl apply -f vald-tensorflow-ingress-filter/k8s
   

   Copy

   The official image only supports amd64.

   NOTE: The example manifest files use BERT from Tensorflow Hub⁠ as the Tensorflow SavedModel. You can change the model by editing k8s/deployment.yaml.

   ...
     - |
       curl -L "https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/4?tf-hub-format=compressed" -o /model/sample.tar.gz  #FIXME
   ...
     env:
       - name: MODEL_PATH
         value: "/model"  #FIXME
       - name: INPUT_TENSOR_NAMES
         value: "name1 name2"  #FIXME
       - name: OUTPUT_TENSOR_NAME
         value: "name"  #FIXME
       - name: REQUEST_TYPE
         value: "int32"  #FIXME
   

   Copy

2. Deploy the Vald cluster with filter gateway

   Please edit the example/helm/values.yaml⁠ in the Vald repository to make vald-filter-gateway available and use it for deployment.

   git clone https://github.com/vdaas/vald.git
   cd vald
   
   vim example/helm/values.yaml
   ---
   ...
   gateway:
   ...
       filter:
           enabled: true
   ...
   agent:
       ngt:
           dimension: 768
   

   Copy

⁠Sample code with vald-client-python⁠

import grpc
import numpy as np
import tensorflow_hub as hub
import tensorflow_text as text
from vald.v1.payload import payload_pb2
from vald.v1.vald import (
    filter_pb2_grpc,
    search_pb2_grpc,
)

# preprocess
preprocess = hub.load('https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/1')
token = preprocess(["TF Hub makes BERT easy!"])
sample = np.vstack([i for i in token.values()])

channel = grpc.insecure_channel("localhost:8081")

# Insert
stub = filter_pb2_grpc.FilterStub(channel)
resize_vector = payload_pb2.Object.ReshapeVector(
    object=sample.tobytes(),
    shape=[3, 128],
)
resize_vector = resize_vector.SerializeToString()

req = payload_pb2.Insert.ObjectRequest(
    object=payload_pb2.Object.Blob(
        id="0",
        object=resize_vector
    ),
    config=payload_pb2.Insert.Config(skip_strict_exist_check=False),
    vectorizer=payload_pb2.Filter.Target(
        host="vald-tensorflow-ingress-filter",
        port=8081,
    )
)
stub.InsertObject(req)

⁠Versions

⁠Dockerfile

The Dockerfile of this image is here⁠.

⁠About Vald Project

The information about the Vald project, please refer to the following:

• Official website⁠
• GitHub⁠

⁠Contacts

We're love to support you! Please feel free to contact us anytime with your questions or issue reports.

• Official Slack WS⁠
• GitHub Issue⁠

⁠License

This product is under the terms of the Apache License v2.0; refer LICENSE⁠ file.