---
title: Using Temporal Cloud With On-Prem Data
description: Ensure data security when using Temporal Cloud. Learn how to customize Temporal to prevent sensitive information from being sent to the cloud.
image: https://www.bitovi.com/hubfs/Using%20Temporal%20Cloud%20With%20On-Prem%20Data.png
---

- ![AI implementation](https://www.bitovi.com/hubfs/AIConsultingIcon.svg)
  
  [AI implementation](https://www.bitovi.com/services/ai-consulting)
- ![Systems engineering](https://www.bitovi.com/hubfs/icon%20-%20backend.svg)
  
  [Systems engineering](https://www.bitovi.com/services/systems-engineering-consulting)
- ![Project Management](https://www.bitovi.com/hubfs/icon%20-%20PM.svg)
  
  [Project Management](https://www.bitovi.com/services/agile-project-management-consulting)
- ![Product Design](https://www.bitovi.com/hubfs/icon%20-%20design.svg)
  
  [Product Design](https://www.bitovi.com/services/product-design-consulting)
- ![Frontend development](https://www.bitovi.com/hubfs/icon%20-%20frontend.svg)
  
  [Frontend development](https://www.bitovi.com/services/frontend-development-consulting)
- [View more
  
  →
  
  ](https://www.bitovi.com/digital-consulting-services)

We're Experts in...

- [JavaScript](https://www.bitovi.com/services/frontend/javascript-consulting)
- [AI training](https://www.bitovi.com/ai-training-for-software-engineers)
- [Angular](https://www.bitovi.com/services/frontend/angular-consulting)
- [Design systems](https://www.bitovi.com/services/axure-figma-migration)
- [React](https://www.bitovi.com/services/frontend/react-consulting)
- [Temporal](https://www.bitovi.com/services/backend/temporal-consulting)
- [React Native](https://www.bitovi.com/services/frontend/react-consulting/react-native)
- [Node.js](https://www.bitovi.com/services/backend/nodejs-consulting)

Showcase

![Yum! Brands](https://www.bitovi.com/hubfs/yum-showcase-link-1.png)

[View case study](https://www.bitovi.com/en/bitovi-yum-case-study)

More Projects

- [![Levi's](https://www.bitovi.com/hubfs/levis.svg)](https://www.bitovi.com/web-application-consulting-work/levis-ecommerce-responsive-redesign)
- [![Christie's International Real Estate](https://www.bitovi.com/hubfs/christies.svg)](https://design.bitovi.com/christies)
- [![BAFS](https://www.bitovi.com/hubfs/bafs.svg)](https://www.bitovi.com/ux-design-consulting/ux-case-studies/bafs-ppp)
- [View more
  
  →
  
  ](https://www.bitovi.com/our-software-consulting-work)

Open Source Tools

We build powerful tools and open source them to support the community.

[See what we've built →](https://www.bitovi.com/open-source)

- [![Blog](https://www.bitovi.com/hubfs/icon%20-%20blog.svg)
  
  BlogWe post about delivering products and solving problems.
  
  ](https://www.bitovi.com/blog)
- [![Partnerships](https://www.bitovi.com/hubfs/Handshake-1.svg)
  
  PartnershipsLearn about Bitovi's technology partners
  
  ](https://www.bitovi.com/partnerships)
- [![Academy](https://www.bitovi.com/hubfs/icon%20-%20academy%20(4).svg)
  
  AcademyFree courses to build delivery skills
  
  ](https://www.bitovi.com/academy)
- [![Open source tools](https://www.bitovi.com/hubfs/icon%20-%20open%20source.svg)
  
  Open source toolsUse or contribute to our community
  
  ](https://www.bitovi.com/open-source)

Let's Connect

- [![Discord](https://www.bitovi.com/hubfs/DiscordLogo.svg)
  
  Discord
  
  ](https://discord.gg/J7ejFsZnJ4)
- [![LinkedIn](https://www.bitovi.com/hubfs/LinkedinLogo.svg)
  
  LinkedIn
  
  ](https://www.linkedin.com/company/bitovi/)
- [![GitHub](https://www.bitovi.com/hubfs/GithubLogo.svg)
  
  GitHub
  
  ](https://github.com/bitovi/)

![Eggbot](https://www.bitovi.com/hubfs/build_assets/bitovi-limbo-cms-react/338/js_client_assets/assets/eggbot-LTGhdSGL.png)

Name *

Work Email *

Phone

What's your project?

Send

### Contact Us

(312) 620-0386contact@bitovi.com

[ Backend ](https://www.bitovi.com/blog/topic/backend) |  November 17, 2023

# Using Temporal Cloud With On-Prem Data

 Ensure data security when using Temporal Cloud. Learn how to customize Temporal to prevent sensitive information from being sent to the cloud.

![Emil Kais](https://www.bitovi.com/hubfs/People/emil_kais.jpg)

 Emil Kais

Share:

[![Twitter](https://www.bitovi.com/hubfs/limbo-generated/_astro/twitter-white.os3xLc3C_Z2nW4or.svg) ](https://twitter.com/intent/tweet?text=) [![Reddit](https://www.bitovi.com/hubfs/limbo-generated/imgs/icons/reddit.png) ](http://reddit.com/submit?url=)

Using cloud services is standard practice for most backend application architectures. When using cloud services, it is important to understand and control what data is leaving your network and being sent to the cloud. Temporal Cloud has great options available to ensure that data sent to and from the cloud is securely encrypted. This post will showcase how Temporal Cloud might interact with your infrastructure by default and how you can customize Temporal to prevent any user or business-related data from being sent to the cloud.

| **Table of Contents** - [Understanding Temporal Cloud](https://www.bitovi.com/blog/using-temporal-cloud-with-on-prem-data#Understanding-Temporal-Cloud) - [Defining Your Own DataConverter Option for Your Temporal Client](https://www.bitovi.com/blog/using-temporal-cloud-with-on-prem-data#Defining-Your-Own-DataConverter-Option) - [Worker and Temporal Client Integration](https://www.bitovi.com/blog/using-temporal-cloud-with-on-prem-data#Worker-and-Temporal-Client-Integration) - [Adding a DataStore to Your DataConverter](https://www.bitovi.com/blog/using-temporal-cloud-with-on-prem-data#Adding-a-DataStore-to-Your-DataConverter) - [Conclusion](https://www.bitovi.com/blog/using-temporal-cloud-with-on-prem-data#Conclusion) |
| --- |

## Understanding Temporal Cloud

Cloud services can be difficult to understand without proper visualization. We created an image to help explain how Temporal Cloud manages data.

![Temporal-Cloud](https://www.bitovi.com/hs-fs/hubfs/Temporal-Cloud.png?width=603&height=308&name=Temporal-Cloud.png)

While Temporal Workers are executing a Workflow, information will be sent into the Temporal Cloud and returned to your server or service upon completion. Now that you can visualize the path of the data, let’s tackle the issue of security and encryption when dealing with information sent to and from the Temporal Cloud.

## Defining Your Own DataConverter Option for Your Temporal Client

Temporal provides a `converter` library that you can import in Go to create your own `DataConverter` to pass as an option to your Temporal Workers and Client. This customized data converter extends a `PayloadCodec` interface with two methods: Encode and Decode. These methods will be called to encode and decode the data that passes through the Temporal Workers, as well as the Temporal Client you create. Here is a high-level (and simplified) look at what it will look like:

![temporal-data-encoding](https://www.bitovi.com/hs-fs/hubfs/temporal-data-encoding.png?width=603&height=308&name=temporal-data-encoding.png)

**Note:** The Codec we use implements `PayloadCodec` which is using `AES Crypt security`

## Worker and Temporal Client Integration

When instantiating the Temporal Client, there is an option to create your own `DataConverter`. You will be creating a method called `NewEncryptionDataConverter`, where you will define your own Codec to be used in the data conversion process.

### NewEncryptionDataConverter Method

This method will return a `*DataConverter` struct, which is from the `go.temporal.io/sdk/converter` library. Pass in your own `Codec` struct, which you define to extend the `PayloadCodec` type. Your `Codec` struct has two methods: Encode and Decode, which we will review later in this post.

See below what this method can look like:

```
func NewEncryptionDataConverter(dataConverter converter.DataConverter, options DataConverterOptions) *DataConverter {
	codecs := []converter.PayloadCodec{
		&Codec{KeyID: options.KeyID},
	}

	return &DataConverter{
		parent:        dataConverter,
		DataConverter: converter.NewCodecDataConverter(dataConverter, codecs...),
		options:       options,
	}
}
```

**Note:** You can find a code sample of this implementation here: [https://github.com/temporalio/samples-go/tree/main/encryption](https://github.com/temporalio/samples-go/tree/main/encryption). Remember `converter` is from the `go.temporal.io/sdk/converter` library.

The key in the codec will be used to encrypt and decrypt the data input to the DataConverter. You may need to change how the key is passed within the `DataConverter` according to your security regulations or preferences. Then, finally, return the `DataConverter` struct that the Temporal Client needs.

The codec struct looks like this:

```
// Codec implements PayloadCodec using AES Crypt.
type Codec struct {
	KeyID string
}
```

### Encode Method

```
// Encode implements converter.PayloadCodec.Encode.
func (e *Codec) Encode(payloads []*commonpb.Payload) ([]*commonpb.Payload, error) {
	result := make([]*commonpb.Payload, len(payloads))
	for i, p := range payloads {
		origBytes, err := p.Marshal()
		if err != nil {
			return payloads, err
		}

		key := e.getKey(e.KeyID)

		b, err := encrypt(origBytes, key)
		if err != nil {
			return payloads, err
		}

		result[i] = &commonpb.Payload{
			Metadata: map[string][]byte{
				converter.MetadataEncoding: []byte(MetadataEncodingEncrypted),
				MetadataEncryptionKeyID:    []byte(e.KeyID),
			},
			Data: b,
		}
	}

	return result, nil
}
```

The code above takes a `protobuf` payload that Temporal defined (`commonpb.Payload`). You retrieve the key needed to encrypt the payload, then the code returns a struct following the `commonpb.Payload` structure.

### Decode Method

```
// Decode implements converter.PayloadCodec.Decode.
func (e *Codec) Decode(payloads []*commonpb.Payload) ([]*commonpb.Payload, error) {
	result := make([]*commonpb.Payload, len(payloads))
	for i, p := range payloads {
		// Only if it's encrypted
		if string(p.Metadata[converter.MetadataEncoding]) != MetadataEncodingEncrypted {
			result[i] = p
			continue
		}

		keyID, ok := p.Metadata[MetadataEncryptionKeyID]
		if !ok {
			return payloads, fmt.Errorf("no encryption key id")
		}

		key := e.getKey(string(keyID))

		b, err := decrypt(p.Data, key)
		if err != nil {
			return payloads, err
		}

		result[i] = &commonpb.Payload{}
		err = result[i].Unmarshal(b)
		if err != nil {
			return payloads, err
		}
	}

	return result, nil
}
```

The decode method takes a payload matching the specific encoding that was provided in the `Encode` method, using the key to decrypt the data and return it in the same `protobuf` style.

**Quick Recap**

You can add a layer of security to your information by passing your custom DataConverter to handle encryption/decryption. This method will be used within Temporal Clients and Temporal Workers. However, data security isn’t the only thing you can do with your `DataConverter`. Although the data is encoded, it is still a large amount of information to send to the Temporal Cloud. Instead, you can use your DataConverter to store the data in a datastore that you host. When you store your data separately, you’ll be sending an encrypted ID to Temporal Cloud, which acts as a reference to the information inside your datastore.

## Adding a DataStore to Your DataConverter

Let’s say you wanted to integrate our own hosted MongoDB (this can be a SQL DB, Redis, S3, or any datastore you choose). Your DataConverter will then store the information during the encoding process and return an encrypted ID to reference the information inside your datastore. At a high level, this is how it will look:

![temporal-datastore](https://www.bitovi.com/hs-fs/hubfs/temporal-datastore.png?width=603&height=308&name=temporal-datastore.png)

The `Encode` and `Decode` methods must be updated to include the MongoDB dependency.

### Updating the Codec Struct

Start by updating the `Codec` struct within the library containing your NewDataConverter method. This way, you can access your MongoDB datastore.

```
// Codec implements PayloadCodec using AES Crypt.
type Codec struct {
	KeyID string
	Db    *mongo.Controller
}
```

### Changes to NewEncryptionDataConverter

Next, change the `NewEncryptionDataConverter` method. You’ll only have to change two lines in the method: creating a MongoDB instance and passing it in as your `Db`field in the `Codec` struct.

```
func NewEncryptionDataConverter(dataConverter converter.DataConverter, options DataConverterOptions) *DataConverter {
	mongoController := mongo.NewMongoController()
	codecs := []converter.PayloadCodec{
		&Codec{KeyID: options.KeyID, Db: mongoController},
	}
	... rest of your code
}
```

In this example, you have a basic MongoDB controller that you define, the code for which can be seen at the end of the blog post.

### Changes to the Encode Method

Here, you’ll create your own UUID, insert the record in your own MongoDB datastore using the UUID as the primary key, and send the encrypted UUID to Temporal Cloud.

```
func (e *Codec) Encode(payloads []*commonpb.Payload) ([]*commonpb.Payload, error) {
	result := make([]*commonpb.Payload, len(payloads))
	for i, p := range payloads {
		//create uniqueID to return
		uuidToCodex := uuid.New()
		dataToInsert := make(map[string]interface{}, 0)
		err := json.Unmarshal(p.Data, &dataToInsert)
		if err != nil {
			return payloads, err
		}

		key := e.getKey(e.KeyID)

		// insert record into db
		dataToInsert["_id"] = uuidToCodex.String()

		e.Db.InsertRecord("codex-data", dataToInsert)

		//return the encrypted uuid back
		b, err := encrypt([]byte(uuidToCodex.String()), key)

		if err != nil {
			return payloads, err
		}

		result[i] = &commonpb.Payload{
			Metadata: map[string][]byte{
				converter.MetadataEncoding: []byte(MetadataEncodingEncrypted),
				MetadataEncryptionKeyID:    []byte(e.KeyID),
			},
			Data: b,
		}
	}

	return result, nil
}
```

### Changes to the Decode Method

Once you decrypt the payload, use the UUID to retrieve the record from the collection in the datastore and then return the data as a `commonpb.Payload` type. It’s crucial to encode your result in `"json/plain"`; otherwise, the Workflow will have encoding errors. What you’ll see returned in your infrastructure is the exact data that we inputted for the workflow.

```
func (e *Codec) Decode(payloads []*commonpb.Payload) ([]*commonpb.Payload, error) {
	result := make([]*commonpb.Payload, len(payloads))
	for i, p := range payloads {
		...code as before
		b, err := decrypt(p.Data, key)
		if err != nil {
			return payloads, err
		}
		result[i] = &commonpb.Payload{}

		// retrieve record by converting []byte into the decrypted UUID
		storedObj, err := e.Db.RetrieveRecord("codex-data", string(b))
		if err != nil {
			return payloads, err
		}

		payload, err := json.Marshal(&storedObj)
		if err != nil {
			return payloads, err
		}
		
    // Metadata, on return the MetadataEncoding is "json/plain"
		result[i] = &commonpb.Payload{
			Metadata: map[string][]byte{
				converter.MetadataEncoding: []byte(converter.MetadataEncodingJSON),
			},
			Data: payload,
		}
	}

	return result, nil
}
```

## Conclusion

This post covered the process of setting up your own DataConverter method in your infrastructure and employing your own encode/decode methods to match your business/regulatory requirements. From there, we reviewed how to reduce the information you send to Temporal Cloud by using your own datastore, sending only an encrypted reference to that data. With this method, you can use Temporal Cloud even if you have regulatory or security requirements that prevent you from sending information outside of your infrastructure. You can find a link to the GitHub repo with the sample code here: [Using Temporal Cloud with On-Prem Data Code Samples](https://github.com/bitovi/temporal-examples/tree/codec-example-go).

### Need more help with Temporal Cloud?

Bitovi is an official [Temporal.io](http://Temporal.io) partner, and we offer free Temporal audits to new clients. Schedule a consultation for expert help with your Temporal implementation.

[![Schedule Your Free Temporal Consultation](https://no-cache.hubspot.com/cta/default/2171535/interactive-142178638135.png) ](https://www.bitovi.com/hs/cta/wi/redirect?encryptedPayload=AVxigLI6Q9aoGw6OVJi43PK48DuIgFzo8Ata9Gb%2FwV2KDhsRniDrRTUpJcTHOYC5qTT7xPoLeGm4xskOticWQdfA4yxtK46ynr9jf2%2Fmr7bSY2yDVeSVxCdvfT2tHwjU3q%2Fg%2FZv9Wuaj%2FPW6Grcd4bVS7PmwlPQRI2VhkQeVaREiKGylYgJpKimEuyhA8aIzkglMfmLdYw5noDXY%2BJkXXfH%2B&webInteractiveContentId=142178638135&portalId=2171535)

[![Tag for temporal](https://www.bitovi.com/hubfs/limbo/icons/tag.svg) temporal ](https://www.bitovi.com/blog/topic/temporal)

 Previous Post

![Intro to Temporal Architecture and Essential Metrics](https://www.bitovi.com/hs-fs/hubfs/Intro%20to%20Temporal%20Architecture%20and%20Essential%20Metrics.png?height=117&name=Intro%20to%20Temporal%20Architecture%20and%20Essential%20Metrics.png) [ Intro to Temporal Architecture and Essential Metrics ](https://www.bitovi.com/blog/intro-to-temporal-architecture-and-essential-metrics)

  

 Next Post

![Sending Transactional Emails with Remix and Amazon AWS SES](https://www.bitovi.com/hs-fs/hubfs/Sending%20Transactional%20Emails%20with%20Remix%20and%20Amazon%20AWS%20SES%20%20.png?height=117&name=Sending%20Transactional%20Emails%20with%20Remix%20and%20Amazon%20AWS%20SES%20%20.png) [ Sending Transactional Emails with Remix and Amazon AWS SES ](https://www.bitovi.com/blog/sending-transactional-emails-with-remix-and-amazon-aws-ses)

```json
{
  "@context" : "http://schema.org",
  "@type" : "Organization",
  "address" : {
    "@type" : "PostalAddress",
    "addressCountry" : "United States",
    "addressLocality" : "Libertyville",
    "addressRegion" : "IL",
    "postalCode" : "60048",
    "streetAddress" : "1134 Pine Tree Lane "
  },
  "alternateName" : "Bitovi",
  "areaServed" : {
    "@type" : "GeoCircle",
    "geoMidpoint" : {
      "@type" : "GeoCoordinates",
      "latitude" : "41.8781",
      "longitude" : "87.6298"
    },
    "geoRadius" : "5000 km"
  },
  "description" : "Bitovi is a UX, UI design and front-end JavaScript development consulting company",
  "email" : "contact@bitovi.com",
  "image" : "https://www.bitovi.com/hubfs/bitovi-logo-x2.png",
  "logo" : "https://www.bitovi.com/hubfs/bitovi-logo-23-1.svg",
  "mainEntityOfPage" : {
    "@id" : "https://www.bitovi.com/blog/using-temporal-cloud-with-on-prem-data",
    "@type" : "WebPage",
    "description" : "Ensure data security when using Temporal Cloud. Learn how to customize Temporal to prevent sensitive information from being sent to the cloud."
  },
  "naics" : "541511",
  "name" : "Bitovi Web App Consulting",
  "sameAs" : [ "https://www.facebook.com/BitoviLLC/", "https://twitter.com/bitovi", "https://www.linkedin.com/company/bitovi" ],
  "telephone" : "312-620-0386",
  "url" : "http://bitovi.com"
}
```

```json
{
  "@context" : "http://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Emil Kais"
  },
  "dateModified" : "November 17, 2023, 3:00:00 PM",
  "datePublished" : "2023-11-17 15:00:00",
  "description" : "Ensure data security when using Temporal Cloud. Learn how to customize Temporal to prevent sensitive information from being sent to the cloud.",
  "headline" : "Using Temporal Cloud With On-Prem Data",
  "image" : {
    "@type" : "ImageObject",
    "url" : "https://www.bitovi.com/hubfs/Using%20Temporal%20Cloud%20With%20On-Prem%20Data.png"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://www.bitovi.com/hubfs/bitovi-logo-23-1.svg"
    },
    "name" : "Bitovi"
  }
}
```