Move DynamoDB Table from Local to AWS Cloud

Table of Contents

1. Introduction
2. DynamoDB Table Schema
3. Prepare the Table Data from Local DynamoDB
4. JSON Data Output
5. Put Request to Add Data
6. Conclusion

Introduction

This blog shows how to move a DynamoDB local table and data to the DynamoDB Web Service. Unlike SQL databases, you can't simply use a backup file to restore to a different environment. So far, at the time of writing, I have not found any other way to move DynamoDB local data to the AWS Web Service. The following sample code will loop through each item and use aws-sdk to put the request to the AWS DynamoDB Web Service.

DynamoDB Table Schema

The following is the sample table schema. You can manually add the table on AWS DynamoDB or use the SDK to create a table programmatically.

var params = {
        TableName : 'MyFakeFriend',
        KeySchema: [
            { AttributeName: "name", KeyType: "HASH" },
            { AttributeName: "age", KeyType: "RANGE" },
        ],
        AttributeDefinitions: [
            { AttributeName: "name", AttributeType: "S" },
            { AttributeName: "age", AttributeType: "S" }
        ],
        ProvisionedThroughput: {
            ReadCapacityUnits: 10,
            WriteCapacityUnits: 10
        }
    };

Prepare the Table Data from Local DynamoDB

This command will scan the entire table MyFakeFriend and output the result to the data.json file.

aws dynamodb scan --table-name MyFakeFriend --endpoint-url http://localhost:8000 > data.json

JSON Data Output

Now, the following is the result when I run the JSON output command. I have 20 items that need to be added to the cloud DynamoDB service.

{
    "Count": 20,
    "Items": [
        {
            "phone": {
                "S": "325-550-4097"
            },
            "age": {
                "S": "16"
            },
            "name": {
                "S": "Clifton Trantow"
            }
        },
        {
            "phone": {
                "S": "923.425.3214 x763"
            },
            "age": {
                "S": "39"
            },
            "name": {
                "S": "Horace Luettgen"
            }
        }
    ],
    "ScannedCount": 20,
    "ConsumedCapacity": null
}

Put Request to Add Data

The following is sample code to loop through DynamoDB local items and put the request to the cloud AWS DynamoDB service.

var AWS = require("aws-sdk");
var fs = require('fs');

function addData() {

    AWS.config.update({
        region: "us-west-2",
        endpoint: "https://dynamodb.us-west-2.amazonaws.com"
      });

    var dynamodb = new AWS.DynamoDB();
    fs.readFile('./data.json', (err, data) => {
        if (err) throw err;
        let jsonData = JSON.parse(data);

        jsonData.Items.forEach(item => {

            var params = {
                TableName: 'MyFakeFriend',
                Item: item
            }

            dynamodb.putItem(params, function(err, data) {
                if (err) {
                    console.error("Unable to add data", JSON.stringify(err, null, 2));
                } else {
                    console.log("Able to add data", JSON.stringify(data, null, 2));
                }
            })
        });
    });
}

addData();

Conclusion

The above approach provides a simple way to migrate DynamoDB data from a local instance to the AWS cloud by scanning the local table, exporting it to JSON, and then programmatically putting each item into the cloud DynamoDB service.