Understanding and Extending Swift 4’s Codable

• Why by SK
• Engineering
• Understanding and Extending Swift 4’s Codable

The Codable protocols (Decodable and Encodable) were introduced to the Swift standard library with Swift 4.

Apple introduces these protocols and their purposes with the following overview:

Many programming tasks involve sending data over a network connection, saving data to disk, or submitting data to APIs and services. These tasks often require data to be encoded and decoded to and from an intermediate format while the data is being transferred. The Swift standard library defines a standardized approach to data encoding and decoding.

The Codable protocols are very easy to implement and there are already many good tutorials available, including official documentation.

As useful as these protocols are, they are missing one piece of functionality that I find is pretty common when working with remote services: the ability to update an existing model object from data. Building this functionality is a great exercise to get to explore the Codable protocols and related types.

If you aren’t already familiar with the Codable protocols and how to implement them, it would be good to check that out first.

Goals

To build updating functionality that works with the existing Codable setup we need three parts:

1. A protocol that model objects can conform to for updating

   protocol DecoderUpdatable {
       mutating func update(from decoder: Decoder) throws
   }

   Done!

2. The ability to use JSONDecoder and other similar decoders to update an object from data

   JSONDecoder().update(&someObject, from: data)

   This will involve a bit more work and take us through “high-level” decoders.

3. The ability to update child objects on a DecoderUpdatable from its update(from:) method

   mutating func update(from decoder: Decoder) throws {
       let container = try decoder.container(keyedBy: Keys.self)
       try container.update(&child, forKey: .child)
   }

   This involves diving deeper into the different types of decoding containers.

High-Level Decoders

When I first started looking into the Codable protocols, I was thrown off by the fact that classes like JSONDecoder and PropertyListDecoder didn’t conform to the Decoder protocol. According to Apple’s (documentation), Decoder is “A type that can decode values from a native format into in-memory representations.” Importantly, a Decoder needs access to some data that it can convert into model objects. JSONDecoder does not have access to any data on its own, and can be used repeatedly for deserializing different objects. It contains the rules for how data is deserialized and can be thought of as a high-level decoder or a decoding format. Since Swift is open source, it’s possible to go look at how JSONDecoder is actually implemented and in doing so you can see that it actually creates a fileprivate class that conforms to Decoder in order to deserialize data. So does PropertyListDecoder.

Since these high-level decoders don’t have access to data and can’t be used directly to deserialize an object it makes sense that they don’t conform to the Decoder protocol. Instead, they represent the “intermediate formats” from Apple’s introduction to Codable. However, this causes an issue for us in implementing DecoderUpdatable, since they don’t just conform to Decoder but the actual decoders they do use are private and there is not a way to access them. They only have a method which takes data, creates a Decoder privately and uses it to create a model object:

func decode<T : Decodable>(_ type: T.Type, from data: Data) throws -> T

What I’d like is a protocol for these decoders that declares its role as a format that can take data and create a Decoder for that data:

protocol DecodingFormat {
    func decoder(for data: Data) -> Decoder
}

From here, we can add an extension to the DecodingFormat protocol so that it can have the same convenient decoding function as before:

extension DecodingFormat {
    func decode<T: Decodable>(_ type: T.Type, from data: Data) throws -> T {
        return try T.init(from: decoder(for: data))
    }
}

In order to get JSONDecoder and PropertyListDecoder to conform to DecodingFormat we need a way to expose the private Decoders that they create. This is a little tricky, but there is a way to do it:

struct DecoderExtractor: Decodable {
    let decoder: Decoder
    
    init(from decoder: Decoder) throws {
        self.decoder = decoder
    }
}

extension JSONDecoder: DecodingFormat {
    func decoder(for data: Data) -> Decoder {
        // Can try! here because DecoderExtractor's init(from: Decoder) never throws
        return try! decode(DecoderExtractor.self, from: data).decoder
    }
}

extension PropertyListDecoder: DecodingFormat {
    func decoder(for data: Data) -> Decoder {
        // Can try! here because DecoderExtractor's init(from: Decoder) never throws
        return try! decode(DecoderExtractor.self, from: data).decoder
    }
}

Now, to achieve our goal of having high-level decoders be able to update objects, we can create an extension on DecodingFormat with a convenience function for updating as well:

extension DecodingFormat {
    func update<T: DecoderUpdatable>(_ value: inout T, from data: Data) throws {
        try value.update(from: decoder(for: data))
    }
}

Containers

To achieve our final goal of updating child objects we are going to need a better understanding of containers. The Decoder protocol has three functions that return different types of containers:

• func container<Key>(keyedBy: Key.Type) – which returns a KeyedDecodingContainer
• func singleValueContainer() – which returns a SingleValueDecodingContainer
• func unkeyedContainer() – which returns an UnkeyedDecodingContainer

The common denominator with the documentation for all of these is that they are a way of accessing the data inside of a Decoder directly. They are abstractions around different formats that data might be stored.

The single value container is well-named and easy to understand, it holds a single value.
The unkeyed container is a collection of values that don’t have keys (like an Array or a Set).
The keyed container is a collection of values that are accessed with keys (essentially a wrapper around a Dictionary).

The KeyedDecodingContainer is the one you’ll see most often (if not always) for Decodable types’ init(from decoder: Decoder) throws method.

SingleValueContainers are used by the decoding primitive types like Ints and Strings for their implementation of init(from decoder: Decoder) throws, which is how they are also able to be Decodable source. UnkeyedDecodingContainer is similarly used for decoding and encoding Arrays and Sets source.

Updating an object from a container

There are a few issues with updating an object from a container:
– In order to call update(from decoder: Decoder) throws, we need a Decoder
– While it is easy to create containers from a Decoder, there is no way to create a Decoder from a container

To solve this problem, we are going to need to create our own Decoder from a container. Since our update method will be called from a KeyedDecodingContainer, we will also have access to the Key that was used to try to access this nested Decoder.

class NestedDecoder<Key: CodingKey>: Decoder {
    let container: KeyedDecodingContainer<Key>
    let key: Key
    
    init(from container: KeyedDecodingContainer<Key>, key: Key) {
        self.container = container
        self.key = key
    }
    
    var userInfo: [CodingUserInfoKey : Any] = [:]
    
    var codingPath: [CodingKey] {
        return container.codingPath
    }
    
    func container<Key>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> where Key : CodingKey {
        return try container.nestedContainer(keyedBy: type, forKey: key)
    }
    
    func unkeyedContainer() throws -> UnkeyedDecodingContainer {
        return try container.nestedUnkeyedContainer(forKey: key)
    }
    
    func singleValueContainer() throws -> SingleValueDecodingContainer {
        // More work needed here
    }
}

By using the functions and properties on the existing container we are able to get most of the way there. There isn’t any kind of nestedSingleValueContainer method on our container, so this will have to be made as well. A SingleValueDecodingContainer is a protocol that just has a decode function for each of the decoding primitive types. To make our own we can continue to leverage the KeyedDecodingContainer and Key that we already have to get the single value for a key, regardless of its type:

class NestedSingleValueDecodingContainer<Key: CodingKey>: SingleValueDecodingContainer {
    let container: KeyedDecodingContainer<Key>
    let key: Key
    var codingPath: [CodingKey] {
        return container.codingPath
    }
    
    init(container: KeyedDecodingContainer<Key>, key: Key) {
        self.container = container
        self.key = key
    }
    
    func decode(_ type: Bool.Type) throws -> Bool {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: Int.Type) throws -> Int {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: Int8.Type) throws -> Int8 {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: Int16.Type) throws -> Int16 {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: Int32.Type) throws -> Int32 {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: Int64.Type) throws -> Int64 {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: UInt.Type) throws -> UInt {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: UInt8.Type) throws -> UInt8 {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: UInt16.Type) throws -> UInt16 {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: UInt32.Type) throws -> UInt32 {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: UInt64.Type) throws -> UInt64 {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: Float.Type) throws -> Float {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: Double.Type) throws -> Double {
        return try container.decode(type, forKey: key)
    }
    
    func decode(_ type: String.Type) throws -> String {
        return try container.decode(type, forKey: key)
    }
    
    func decode<T>(_ type: T.Type) throws -> T where T : Decodable {
        return try container.decode(type, forKey: key)
    }
    
    func decodeNil() -> Bool {
        return (try? container.decodeNil(forKey: key)) ?? false
    }
}

We are also losing the userInfo from the original Decoder, but there isn’t much we can do about that since our container does not have access to its Decoder or its userInfo. We can allow this to be explictly passed in the NestedDecoder‘s init method in case this is important.

So our final NestedDecoder code looks like this:

class NestedDecoder<Key: CodingKey>: Decoder {
    let container: KeyedDecodingContainer<Key>
    let key: Key
    
    init(from container: KeyedDecodingContainer<Key>, key: Key, userInfo: [CodingUserInfoKey : Any] = [:]) {
        self.container = container
        self.key = key
        self.userInfo = userInfo
    }
    
    var userInfo: [CodingUserInfoKey : Any]
    
    var codingPath: [CodingKey] {
        return container.codingPath
    }
    
    func container<Key>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> where Key : CodingKey {
        return try container.nestedContainer(keyedBy: type, forKey: key)
    }
    
    func unkeyedContainer() throws -> UnkeyedDecodingContainer {
        return try container.nestedUnkeyedContainer(forKey: key)
    }
    
    func singleValueContainer() throws -> SingleValueDecodingContainer {
        return NestedSingleValueDecodingContainer(container: container, key: key)
    }
}

Now we’re able to add an extension to KeyedDecodingContainer to update model objects conforming to DecoderUpdatable, meeting the last of our goals:

extension KeyedDecodingContainer {
    func update<T: DecoderUpdatable>(_ value: inout T, forKey key: Key, userInfo: [CodingUserInfoKey : Any] = [:]) throws {
        let nestedDecoder = NestedDecoder(from: self, key: key, userInfo: userInfo)
        try value.update(from: nestedDecoder)
    }
}

Conclusion

The Codable protocols can feel like magic, especially because Xcode is able to generate your CodingKeys, init(from:) and encode(to:) functions automatically. With a little work we were able to make DecoderUpdatable work with both high-level decoders and with low-level containers. We can take this exercise further by adding conveniences for ManagedObjects or custom Decoders for different formats. Now that we’re familiar with the Codable protocols and types, the curtain has been pulled back and it’s not as magic as it seems.

The SK Engineering Team