Interactive retailing on GCP with IoT: Containers at the Store Edge

22/10/19, by Gavin Hamill

Hello!

I’d like to showcase today some of the work that we’ve been doing for one of our most interesting clients, Perch.

By collaborating with Perch, companies can deliver brand messages for each product on the counter magically, the moment a customer touches a product using computer vision. If you were to spend a few seconds on their homepage; the video there will immediately show you why retailers like Macy’s, Lenovo and Neiman Marcus have chosen to partner with Perch.

Perch’s interactive retail displays include embedded IoT technology that unites digital content with physical products, delivering highly personalized product messaging. This application of product engagement marketing is applicable to a plethora of retail categories, including fragrance, beauty, fashion, and more.

In every case, physically interacting with the product will cause the screen to respond and give additional product information. Perch’s system records data about every interaction allowing retailers to receive detailed reports for each product on how often it was viewed, for how long, and which interactions led to a sale. Now, for the first time, companies can have insight into the middle of the funnel shopper behavior around product discovery and purchase in-store while allowing the shopper to discover products in-store more naturally.

Deployments

The deployment for Perch is made particularly interesting because while we’ve delivered a series of Cloud Functions in GCP, there’s still a need to identify, update and troubleshoot dozens of end devices at each store location. This presents us with a great opportunity to standardise the deployment process for each of those devices, thus reducing the ongoing support effort.

What’s in the Cloud?

The overall system architecture includes a series of Cloud Functions in GCP managing:

  • registration of new devices (bootstrap)
  • verifying token authenticity
  • sending analytics data
  • control messages

Other Google Cloud Platform services we’re making use of include:

  • Firestore acts as the source of truth of authorised devices.
  • IoT Core allows us to keep better track of the online status of each device
  • Container Registry grants private access to the Docker images for each store
  • Stackdriver gives us the big picture on resource usage across the estate
  • Cloud Storage as a Debian APT repository for storing new versions of store-specific data

Finally a set of Kubernetes Deployments running in GKE are responsible for:

  • MongoDB managing user dashboards
  • InfluxDB recording device interactions
  • OpenVPN server
  • Grafana, Prometheus and AlertManager
  • Diagnostics
  • Jenkins for managing deployments
  • several ancillary services

The devices themselves are regular PC hardware connected to 3D cameras, IR sensors, and touch-sensitive displays. As strong supporters of open source technologies, we were pleased when Perch were running Ubuntu 18.04 LTS on each device.

We automated a manual provisioning system into a ‘single script’ bootstrap launched from the Cloud or from a USB stick (in-store Wi-Fi is rarely fast or reliable!) approach using Ansible 2.8 and a Cloud based dynamic inventory which bundles all the dependencies for successful provisioning:

  • .deb files for a working Python 3 installation
  • Python 3 modules necessary for the Ansible provisioning to succeed.
  • the perch-ansible Debian package containing data unique to this store
  • a set of Docker images as tarballs (~2 GB) - mostly pre-rendered animations

Docker-at-Edge

To simplify the development and deployment processes, the software being used on each device is split into a number of services, each of which runs in its own Docker container. Since the Perch stack is reliant on displaying smooth video and collecting user input from a set of smart sensors, this required a little fun with the DISPLAY variable, shared devices, GPU acceleration, and ensuring that the Docker containers were running as the same user that Ubuntu’s GDM performs an auto-login to.

Authentication

Authentication in the cloud can be a complex process, and we bumped into an entertaining ‘chicken and egg’ situation. An earlier version of the deployment system had a hard-coded Service Account JSON token that each device used to register itself into IoT Core. This was a security smell and we wondered how we could move to a situation where a device could be rolled out without embedded credentials.

Since each of the devices are manufactured to Perch’s specification, we use device-specific data to precalculate a set of sha256 hashes so that when the bootstrap process runs and sends a hostname and a hash to the device registration Cloud Function, we can be confident that the device requesting to connect is genuine Perch hardware.

The device registration Cloud Function generates an X.509 public/private key pair and sends this back to the device in addition to registering that device in IoT Core. This allows the device to generate a JWT which in turn is used to request an OAuth2 access token which is used by the Docker daemon to access the images in Cloud Registry.

Metadata server at EDGE

As we deploy new functionality, Jenkins uses IoT Core to send a message to each device in a group that it should install a given version of the perch-ansible package. This in turn will cause Ansible to re-run on those devices, (the development process uses both real hardware devices and Vagrant boxes running Ubuntu 18.04) updating the Docker and satellite services.

The authentication problem continues! We store those packages as private objects in Google Cloud Storage, and we are using an adaptation of APT transport so that when we update /etc/apt/sources.list.d with our own repo line, it uses gs://our-bucket-name-for-this-environment/assets/ rather than https://

The use-case for the APT transport is typically on Compute instances inside the Google Cloud Platform, but we’re running it on standalone devices in retail stores. We needed a solution!

We chose to solve the problem by inventing a local metadata service which would listen on the well-known metadata IP address 169.254.169.254. In this way the APT transport can work using DEFAULT CREDENTIALS without a single line of code change! A simplified dev-only version of the metadata service is included here:

import json
from http.server import HTTPServer, BaseHTTPRequestHandler

def set_virtual_ip():
    bashCommand = "ip a a 169.254.169.254/32 dev lo"
    process = subprocess.Popen(bashCommand.split(), stdout=subprocess.PIPE)
    output, error = process.communicate()
    print(output, error)

serviceAccountIdResponse = {
        "aliases": ["default"],
        "email": "nameofyourSA@your-project-name.iam.gserviceaccount.com",
        "scopes": [
            "https://www.googleapis.com/auth/devstorage.read_only"
            ]
        }

class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):

    def getAccessTokenResponse(self):
        with open('PATH_TO_TOKEN/token', 'r') as tokenfile:
            token = tokenfile.read()

        accessTokenResponse = {
                "access_token":token,
                "expires_in":300,
                "token_type":"Bearer"}
        return accessTokenResponse

    def do_GET(self):
        self.server_version = 'Metadata Server for VM'
        self.sys_version = ''
        print(self.path)
        if self.path == '/':
            self.send_response(200)
            self.send_header('Metadata-Flavor', 'Google')
            self.send_header('Content-type', 'application/text')
            self.end_headers()
            self.wfile.write(b'0.1/\ncomputeMetadata/\n')
        elif self.path.startswith('/computeMetadata/v1/project/project-id'):
            self.send_response(200)
            self.send_header('Metadata-Flavor', 'Google')
            self.send_header('Content-type', 'application/text')
            self.end_headers()
            self.wfile.write(b'our-project-name')
        elif self.path.startswith('/computeMetadata/v1/instance/service-accounts/default/?recursive=true') or self.path.startswith('/computeMetadata/v1/insta
nce/service-accounts/{}/?recursive=true'.format(serviceAccountIdResponse['email'])):
            self.send_response(200)
            self.send_header('Metadata-Flavor', 'Google')
            self.send_header('Content-type', 'application/json')
            self.end_headers()
            self.wfile.write(json.dumps(serviceAccountIdResponse, separators=(',', ':')).encode('utf8'))

        elif self.path.startswith('/computeMetadata/v1/instance/service-accounts/default/token'):
            self.send_response(200)
            self.send_header('Metadata-Flavor', 'Google')
            self.send_header('Content-type', 'application/json')
            self.end_headers()
            self.wfile.write(json.dumps(self.getAccessTokenResponse(), separators=(',', ':')).encode('utf8'))


set_virtual_ip()
httpd = HTTPServer(('169.254.169.254', 80), SimpleHTTPRequestHandler)
httpd.serve_forever()

It’s been working really well, and we’d recommend this approach for similar problems in future. We use it not only for APT transport but also for any Google API based service running on the devices.

There’s still plenty to do for Perch, and we can’t wait to see what new features they want to move forward with, as we know we’ll enjoy implementing them!

22/10/19 Interactive retailing on GCP with IoT: Containers at the Store Edge, by Gavin Hamill

comments powered by Disqus