Module 3: DynamoDB Delete

Learning Objectives

  • Implement functionality that deletes an item from a provided DynamoDB table by partition key
  • Implement functionality that deletes an item from a provided DynamoDB table by partition and sort key
  • Implement functionality that conditionally deletes an item from a provided DynamoDB table
  • Design and implement functionality to deactivate rather than delete an item from a provided DynamoDB table in order to prevent loss of information

Introduction to DynamoDB Delete Operations

In the previous unit, you were introduced to one of Amazon's database services: DynamoDB. You learned how to annotate a Plain Old Java Object (POJO) to map a table item to Java, and use the DynamoDBMapper methods load() and save() to get, create, and update items in a table. In this reading we're going to discuss how to use the DynamoDBMapper method delete() to remove items from a table.

Deleting an Item with a Partition Key

Remember your Shoes table? It was created to help organize your closet by keeping track of the shoes you own and where they're stored. Each shoe has a unique id (shoe_id), a location of where they're stored (cubby_location) and then attributes about their color, style, and occasion. The table uses only a partition key, which is shoe_id.

shoe_id cubby_location color style occasion
SN01 1 grey sneaker athletic
BO01 2 grey boot work
SN02 3 black sneaker casual

Your grey athletic sneakers have gotten pretty worn down and it's time to throw them out (shoe_id = "SN01"). Since you're throwing the shoes out, you want to delete this item from your table to reflect the change. We can use the DynamoDBMapper delete() method to remove the shoes from your table.

Let's review the fully annotated Shoe POJO we created in the previous unit:

@DynamoDBTable(tableName = "Shoes")

public class Shoe {
    private String shoeId;
    private int cubbyLocation;
    private String color;
    private String style;
    private String occasion;

    @DynamoDBHashKey(attributeName = "shoeId")
    public String getShoeId() {
        return shoeId;
    }

    public void setShoeId(String shoeId) {
        this.shoeId = shoeId;
    }


    @DynamoDBAttribute(attributeName = "cubbyLocation")
    public int getCubbyLocation() {
        return cubbyLocation;
    }

    public void setCubbyLocation(int cubbyLocation) {
        this.cubbyLocation = cubbyLocation;
    }


    @DynamoDBAttribute(attributeName = "color")
    public String getColor() {
        return color;
    }

    public void setColor(String color) {
        this.color = color;
    }


    @DynamoDBAttribute(attributeName = "style")
    public String getStyle() {
        return style;
    }

    public void setStyle(String style) {
        this.style = style;
    }


    @DynamoDBAttribute(attributeName = "occasion")
    public String getOccasion() {
        return occasion;
    }

    public void setOccasion(String occasion) {
        this.occasion = occasion;
    }
}

Using the above Shoe POJO, deleting SN01 from the Shoes table with the DynamoDBMapper delete() method looks like the following:

DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
Shoe shoe = new Shoe();
shoe.setShoeId("SN01");
mapper.delete(shoe);

The first line creates an instance of DynamoDBMapper, just like we did in the load() and save() methods. Then we create a new instance of the Shoe class, shoe, and set shoeId to "SN01". The final line of code uses our new mapper instance to call the delete method, which accepts shoe.

Like the save() method, the delete() method accepts an object, which in this case is our new instance of shoe. The delete() method, however, only uses the object's key values to determine which item to delete. Since our Shoes table only uses a partition key, shoeId, the only value we need to set before passing in the object is shoeId. You do not need to call load() to retrieve the other values of the Shoe item from DynamoDB before deleting it. You can pass in an object that has more values set, but at the bare minimum the key values must be set.

Deleting an Item with a Partition + Sort Key

In the previous unit we also had a Songs table, which keeps track of the songs on our playlist. Each song has an artist (partition key), a song title (sort key), and then attributes about their genre, year released, and whether the song has been favorited. The table uses a composite primary key of artist and song title.

Let's look at the Songs table:

artist song_title genre year favorited
Black Eyed Peas I Gotta Feeling pop 2009 false
Linkin Park Numb rock 2003 true
Black Eyed Peas Pump It pop 2005 true
Eminem Not Afraid rap 2010 false
Daddy Yankee Gasolina latin pop 2004 true
Linkin Park In the End rock 2000 true

The song "I Gotta Feeling" by the Black Eyed Peas is overplayed and not one of your favorites anyways, so you've decided to delete it from your playlist ("Black Eyed Peas", "I Gotta Feeling"). Since you're deleting the song from your playlist, you want to delete the item from your table to reflect the change. We can once again use DynamoDBMapper.delete() to remove the song from the table.

Let's review the fully annotated Song POJO we created in the previous unit:

@DynamoDBTable(tableName = "Songs")
public class Song {
    private String artist;
    private String songTitle;
    private String genre;
    private int year;
    private boolean favorited;

    @DynamoDBHashKey(attributeName = "artist")
    public String getArtist() {
        return artist;
    }

    public void setArtist(String artist) {
        this.artist = artist;
    }

    @DynamoDBRangeKey(attributeName = "song_title")
        public String getSongTitle() {
        return songTitle;
    }

    public void setSongTitle(String songTitle) {
        this.songTitle = songTitle;
    }

    @DynamoDBAttribute(attributeName = "genre")
        public String getGenre() {
        return genre;
    }

    public void setGenre(String genre) {
        this.genre = genre;
    }

    @DynamoDBAttribute(attributeName = "year")
        public int getYear() {
        return year;
    }

    public void setYear(int year) {
        this.year = year;
    }

    @DynamoDBAttribute(attributeName = "favorited")
        public boolean isFavorited() {
        return favorited;
    }

    public void setFavorited(boolean favorited) {
        this.favorited = favorited;
    }
}

Using the above Song class, deleting "I Gotta Feeling" by the Black Eyed Peas with the DynamoDBMapper delete() method looks like this:

DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
Song song = new Song();
song.setArtist("Black Eyed Peas");
song.setSongTitle("I Gotta Feeling");
mapper.delete(song);

This code snippet looks very similar to the code snippet for deleting a shoe from our Shoes table. The first line creates an instance of DynamoDBMapper, then we create a new instance of the Song class, song, and set the artist to "Black Eyed Peas" and the song title to "I Gotta Feeling". The final line of code uses our new mapper instance to call the delete() method, which accepts song.

As we discussed in the previous section, the delete() method accepts an object, which in this case is our new instance of song. The delete() method, however, only uses the object's key values to determine which item to delete. Our Songs table uses a composite primary key, so we need to make sure we set values for both the artist and the songTitle. As we stated previously, you can have more values than just the keys set, but at the bare minimum, the key values must be set for the delete() method to know which item to remove.

Idempotent Delete

What happens if you attempt to delete an item that doesn't exist in the table? Nothing! If DynamoDBMapper can't find the item, it therefore has nothing to delete, but there are no errors or indicators that anything was different than if the item had been deleted. This is a purposeful feature. Delete is an idempotent operation, meaning that if you run the operation multiple times on the same item, the response is unchanged. If the method gave a different response based on whether the item existed or not, the response would change, and the method wouldn't be idempotent.

It is possible to specify conditions so that the delete() method indicates whether there was an item to delete, but you don't need to worry about that now. For now, just understand that the response of the delete() method will remain the same whether or not there is an item to remove.

DynamoDB provides several ways to delete data from your tables. In this module, we focus on using the DynamoDBMapper.delete() method to remove items from DynamoDB tables. Understanding how to properly delete data is crucial for maintaining your database and ensuring data integrity throughout your application.

Delete operations in DynamoDB are designed to be idempotent, which means that you can execute the same delete operation multiple times without changing the result beyond the initial application. This is important for handling retries and error recovery in distributed systems.

Conditions for Deleting

In the previous reading, we learned how to remove items from a table using the DynamoDBMapper delete() method. Having the ability to delete any item from a table can cause issues, however, and it can be useful to set up conditions to limit deletion. In this reading, we'll be discussing how to set restrictions on deleting items and designing your table attributes in a way that you don't need to delete items at all.

Conditional Deletes

To prevent loss of data, you may want to restrict which items can be removed to prevent accidental or premature deletion. For example, imagine you're starting an online clothing store to sell some of your old clothes. Each item has a unique id (partition key), a clothing type, color, price, and current status. The value of the status attribute is either "not sold", "sold", "shipped", or "received".

The ClothingItems table:

id clothing_type color price status
je_398020b8 jeans black 32.00 sold
sh_39802342 shirt blue 10.53 not sold
bo_3980264e boots brown 50.45 received
ts_39802bd0 tshirt green 8.25 shipped

The code for the Clothing POJO looks like the following:

@DynamoDBTable(tableName = "ClothingItems")

public class Clothing {
    private String id;
    private String clothingType;
    private String color;
    private Double price;
    private String status;

    @DynamoDBHashKey(attributeName = "id")
    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }


    @DynamoDBAttribute(attributeName = "clothing_type")
    public String getClothingType() {
        return clothingType;
    }

    public void setClothingType(String clothingType) {
        this.clothingType = clothingType;
    }


    @DynamoDBAttribute(attributeName = "color")
    public String getColor() {
        return color;
    }

    public void setColor(String color) {
        this.color = color;
    }


    @DynamoDBAttribute(attributeName = "price")
    public Double getPrice() {
        return price;
    }

    public void setPrice(Double price) {
        this.price = price;
    }


    @DynamoDBAttribute(attributeName = "status")
    public String getStatus() {
        return status;
    }

    public void setStatus(String status) {
        this.status = status;
    }
}

To ensure you're not accidentally deleting values before they've been received by the buyer, you decide to implement a condition that an item can only be deleted when status = "received". In order to create this condition, we need to use the DynamoDBDeleteExpression class to create a new delete expression and pass it in to our delete() method along with the Clothing object.

The code to delete the pair of boots in our table (id = "bo_3980264e"), which have already been received by the buyer, is as follows:

DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
Clothing boots = new Clothing();
boots.setId("bo_3980264e");

try {
    DynamoDBDeleteExpression deleteExpression = new DynamoDBDeleteExpression();
    Map expected = new HashMap<>();
    expected.put("status", new ExpectedAttributeValue(new AttributeValue("received")));
    deleteExpression.setExpected(expected);
    mapper.delete(boots, deleteExpression);
} catch (ConditionalCheckFailedException e) {
    System.out.println(e);
}

This is doing a few things, so let's walk through it line by line.

The first three lines are familiar: we create our mapper instance, create a new Clothing instance, boots, and then set the id for boots since id is the partition key of the table.

We then enter in a try statement where we create a new instance of DynamoDBDeleteExpression.

DynamoDBDeleteExpression deleteExpression = new DynamoDBDeleteExpression();

This class enables adding options to a delete operation. In our example, we are adding the option to delete only if an attribute has a particular value. To learn more about DynamoDBDeleteExpression, see the DynamoDBDeleteExpression javadoc.

The following line creates a new HashMap called expected and puts a new entry in the map, specifying that the status attribute should have the value of "received" in order to be deleted.

Map expected = new HashMap<>();
expected.put("status", new ExpectedAttributeValue(new AttributeValue("received")));

In this code snippet, the HashMap key is the String name of the DynamoDB item attribute. The value is an ExpectedAttributeValue. The ExpectedAttributeValue accepts a new AttributeValue, which is the value you expect the attribute to be to meet your delete condition. Since we only want to delete items when the attribute status has a value of "received", the key in our HashMap is "status" and the value is the ExpectedAttributeValue, "received". You can learn more about ExpectedAttributeValue in the ExpectedAttributeValue javadoc. The next line of code sets the expected delete expression of our previously instantiated DynamoDBDeleteExpression, deleteExpression, to the HashMap, expected.

deleteExpression.setExpected(expected);

The line after that is where we finally use DynamoDBMapper.delete() to attempt to delete the item.

mapper.delete(boots, deleteExpression);

We're passing in both our previously created boots object and the deleteExpression, which has the delete condition we're using to attempt to delete the item.

The final section is a catch statement, which catches a ConditionalCheckFailedException if the item we attempt to delete does not meet the expected condition.

catch (ConditionalCheckFailedException e) {
    System.out.println(e);
}

Returning to our ClothingItems table, if we attempt to delete the brown boots, we will be successful, because the value of status is "received", therefore meeting our delete condition.

id clothing_type color price status
je_398020b8 jeans black 32.00 sold
sh_39802342 shirt blue 10.53 not sold
bo_3980264e boots brown 50.45 received
ts_39802bd0 tshirt green 8.25 shipped

If we attempt to delete the jeans, however, the item will not be deleted, and instead will catch a ConditionalCheckFailedException, because the value of status is "sold", not "received".

id clothing_type color price status
je_398020b8 jeans black 32.00 sold
sh_39802342 shirt blue 10.53 not sold
bo_3980264e boots brown 50.45 received
ts_39802bd0 tshirt green 8.25 shipped

Active vs. Inactive Items

Sometimes it's better not to delete items from a table, so that you don't lose any information. In the previous unit we discussed the ClubMembers table, which keeps track of members in a club.

id active age last_name year_joined
dac041be true 25 Jackson 2016
dac045a6 true 27 Juan 2015
dac046f0 false 27 Santos 2018
dac04a56 true 21 Wei 2019

The ClubMembers table includes an active attribute, which indicates each member's current status in the club. If a ClubMember decides to deactivate their membership for a year or two, then active = "false", but they're not deleted from the table. This means that if they decide to reactivate their membership at any point, all their information is still there! If a member was deleted and then decided to rejoin the club, you'd have to re-enter all their information. If there's a chance the item's information may be relevant later, even if it currently isn't, it can be better to list the item as "inactive" instead of deleting it from the table.

job_id completed_by year_completed hours_taken
JO7895 Arnav Desai 2017 10
JO1543 Carlos Salazar 2018 16
JO9457 Li Juan 2016 5
JO1154 Martha Rivera 2019 8

Thinking back to the Task class you set up in the previous activity, you delete each task after it's completed. A few months later, one of the systems that was created is having an error and you want to see who completed the relevant task so they can fix it. Since the task was deleted, however, you're not sure who completed it. To avoid this issue in the future, you decide to keep track of whether a task is "active" (needs to be completed) or "inactive" (completed) instead of deleting it.

job_id status completed_by year_completed hours_taken
JO7895 inactive Arnav Desai 2017 10
JO1543 inactive Carlos Salazar 2018 16
JO9457 inactive Li Juan 2016 5
JO1154 inactive Martha Rivera 2019 8
JO4387 active Jorge Souza 15
JO9807 active Sofía Martínez 3

By keeping track of active vs. inactive you're not losing any data and can still access the item even if it's not in use. You can make the "active" attribute the sort key for your table and use the attribute to sort your table and only retrieve active items. In the ClubMembers table, the active value is a boolean because it isn't a key value. Key values, however, cannot be of type boolean, so for our TaskTracker table, we can use the String type instead!

In a future lesson you'll learn how to query a table and return multiple active items, but for now you can use the load() method to only retrieve an item if it's active.

DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
Task taskActive = mapper.load(Task.class, "JO4387", "active");

This code snippet will load task "JO4387" only if the attribute status is currently active, meaning that the job still needs to be completed. Since task "JO4387" is currently active, the item is returned by the load() method.

The following code snippet, however, will retrieve a null value, because the task "JO1154" is currently inactive.

DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
Task taskInactive = mapper.load(Task.class, " JO1154", "active");

Since keeping track of active vs. inactive items allows us to still understand whether an item is currently in use or not, you will more commonly use this table design than deleting items from a table. Deleting items from a table, even if you are doing so with a condition, can lead to loss of data that you may need to access at some point. Think carefully about the design of your tables so that you can keep track of active items instead of having to delete items when they're no longer in use.

Guided Project

Invites

Setup Your Sprint 21 Challenge Repo

This Sprint culminates in a Sprint Challenge project. You should begin by forking and cloning the Sprint Challenge starter repo:

Sprint 21 Challenge Starter Repo

This will be your project repo for Sprint 21-23.

This resource is also visible under the Sprint Challenge section of the course page. After each module, you will be assigned a mastery task with instructions on adding to or modifying the starter code for the challenge. Upon completion of all mastery tasks, the Sprint Challenge project will be complete and ready for you to submit to CodeGrade. The CodeGrade submission page is available under the Sprint Challenge section on the modules page.

Mastery Task 1: Killing Me Softly

Milestone 1: Create class and sequence diagrams

One of the action items from the design review was to add a class diagram with the existing classes in the service. Create a class diagram of the classes in the following packages and how they relate to each other:

  • com.amazon.ata.kindlepublishingservice.activity (except ExecuteTctActivity.java)
  • com.amazon.ata.kindlepublishingservice.clients
  • com.amazon.ata.kindlepublishingservice.dao
  • com.amazon.ata.kindlepublishingservice.dynamob.models
  • com.amazon.ata.kindlepublishingservice.exceptions
  • com.amazon.ata.kindlepublishingservice.metrics (except MetricsConstants.java)

The diagram does not need to include any classes in the following packages:

  • com.amazon.ata.kindlepublishingservice.converters
  • com.amazon.ata.kindlepublishingservice.dagger
  • com.amazon.ata.kindlepublishingservice.publishing

The DynamoDB model classes in the diagram must include all fields and types used to model the DynamoDB table, and which fields will represent the partition and (if any) sort key. You do not need to provide any other annotations, but be sure to indicate the Java type. In each provided model, we've chosen to use an enum to represent the value of an attribute. You'll see this annotation: @DynamoDBTypeConvertedEnum on the getter of the attribute. This tells the DynamoDBMapper to convert the enum to a string value when storing it in the table, but allows your Java code to restrict the attribute to a set of allowed values. Use the enum type in your diagram.

Use the src/resources/mastery-task1-kindle-publishing-CD.puml to document the class diagram.

Next, create a sequence diagram for the API we'll be working on in milestone 2, RemoveBookFromCatalog. You can use the existing sequence diagrams in the design document as a starter. Include any error handling the APIs are expected to perform, as well as their interactions with other classes.

Update src/resources/mastery-task1-remove-book-SD.puml with a sequence diagram for your planned implementation of the RemoveBookFromCatalog operation.

Recall: We can use PlantUML's alt syntax to represent if/else cases for validation.

Recall: We can add the @DynamoDBHashKey and @DynamoDBRangeKey annotations to the class diagram.

Milestone 2: Implement RemoveBookFromCatalog

We already got a head start on this. You'll need to now add some logic to do a soft delete. We don't want to lose previous versions of the book that we have sold to customers. Instead, we'll mark the current version as inactive so that it can never be returned by the GetBook operation, essentially deleted.

We'll need to update our CatalogDao to implement this "delete" functionality and use that in our Activity class.

When writing unit tests for your new logic in CatalogDao, we encourage you to use ArgumentCaptors. To see one in action in the project take a look at the getBookFromCatalog_oneVersion_returnVersion1 unit test in the CatalogDaoTest class.

We've generated some catalog data and put it in your CatalogItemVersions DynamoDb table. You can find a book there to remove, or feel free to add any additional test data.

Run MasteryTaskOneDesignTests to make sure all tests for this task are passing.

Exit Checklist:

  • Your've created your RemoveBookFromCatalog sequence diagram
  • You've implemented RemoveBookFromCatalog's functionality
  • MasteryTaskOneDesignTests passes