This box will disappear when printing mkdocs-print-site-plugin

This page has combined all site pages into one. You can export to PDF using File > Print > Save as PDF.

See also export to PDF and export to standalone HTML.

Introduction

This autogenerated website stores all the relevant artifacts produced to document the process of designing the system following a Domain Driven approach as well as the final architecture of the solution.

Project Proposal

Members

• Samuele Burattini: samuele.burattini@studio.unibo.it
• Luca Deluigi: luca.deluigi3@studio.unibo.it
• Francesco Dente: francesco.dente@studio.unibo.it
• Simone Magnani: simone.magnani4@studio.unibo.it

Scenario

The group is committed to design a microservice system that collects and merge e-scooter related data to create a digital twin of the physical entity. The collected data will be:

• Battery percentage
• Position and speed in real-time
• Activity status (rented, idle, etc.)
• Lock state
• Availability for renting
• Any other relevant data discovered while developing

The goal is to maximize the coverage of the renting service by analyzing users and scooters behaviour to map the more requested areas and which kind of trips the users do most in different areas.

To also make use of the power of the digital twins onto their corresponding physical device a mechanism of battery conservation is put in place to guarantee a long lasting battery life by limiting speed when the battery is too low and a remote locking and disabling mechanism that activates when the scooter is outside of the recovery service area or when the battery is below the minimum threshold to guarantee the device not to turn off.

Monitoring will be made possible through an admin dashboard that show the overall state of all the devices in real-time.

Course requirements

The system will be designed with a domain driven approach. In detail the team will have:

• an initial knowledge crunching session (with fake domain experts)
• the definition, mantain and use of the ubiquitous language
• a requirement analysis, carried out through user stories and use-case diagrams
• bounded context partitioning and definition of a context-map
• domain models for each relevant bounded context

Apart from that the project will follow a development process based on continuos integration & deployment for most of the system components. The tools used will be:

• Dotnet CLI for build automation
• Github Actions for continuous integration
• Github Pages for the hosting of the automatically generated documentation website
• Microsoft Azure as a target platform for the deployment of the whole system
• Quality Assurance tools to be defined during developement (linter, automatic testing etc.)

Domain Exploration

Business Problem Analysis

In order to kick-off the knowledge crunching process, the first discussion with the stakeholders is focused on the identification of the business needs. The objective is to derive how it might be possible to create a software system that could maximise the company business gain/value.

While doing so the analysts start developing the Ubiquitous Language in parallel, to write down the definition of the domain terminology given that they have a clear understanding of the vocabulary in the domain space.

Stakeholders Interview

What is the problem that you're trying to solve by adopting a new software solution?

We would like to increase the average number of E-scooter rents in a day.

What are the possible factors that influence that?

Electric scooters have batteries that run out during the day, we often are not able to recharge them until night so we progressively lose available scooters as the day goes on.

Do you think there is a way to save some battery during the day to make the scooter last longer?

Speed is the main factor that drain the scooter battery. If you want to go fast the motor consumes more energy, maybe speed control might be a thing to look into. Also it's very important that scooters with a low level of battery don't turn off because we can lose the tracking.

About that, should the users be prevented from picking up very low battery scooters?

Yes we would like that so we don't encounter problems. Also we would like to be very upfront with our customers about the information about the scooter when they need to choose one: if you need to do a long trip the worst thing that can happen is to have your scooter stop at some point and need to find another one.

Why is it hard to recover and recharge a scooter?

We scatter scooters around the city and then let people wander and drop them off wherever they like, sometimes they can be pretty far away from each other and make the recovering process tedious and costly to do during the day.

If it was easier do you think you would be able to recharge some drained scooter and put them back as available?

Yes it should be doable, the scooters take about 3 hours to recharge and then they are ready to go again.

What else do you think can improve the number of rents?

We noticed that not all scooters are used in the same way. In some areas they are quickly depleted whilst in some others they stay basically fully charged and sometime we recover them in the same spot where we dropped them. Also the same scooter can travel a long distance either by a lot of short trips or very long ones depending on the area but we have little data on that.

How do you choose where to place your scooters?

Right now we're going by trial and error basically, and then we try to note down which places are the best. If one day we see that all the scooters in a spot have been used we try to put some more in the next one. Our objective is that if you need a scooter you should never have to walk too much to find one.

Impact Mapping

The result of the interview is summed up into an impact map, identifying the root problem and different possible features of interest that raised from the discussion. Theese can be later further developed through the analysis of the usecases and evolve in the system requirements.

Ubiquitous Language

E-Scooter service

A service that allows customers to rent electric scooters placed in public areas for a trip within the boundaries of an area of service.

Electric Scooter

Aliases: e-scooter, scooter
Motorized vehicle with two wheels, usable by a person standing on it. It's equipped with some sensors and other devices.

E-Scooter equipment

Electric engine

Uses power from the battery to power the front wheel spin.

Wheels and tires

The scooter runs on two wheels with air-inflated tires; the front wheel containes a rotating electric engine.

Battery

Scooter power supply; rechargeable; electric; has finite capacity, power...

GPS sensor

Global Positioning System; estimates a position on Earth by negotiating it with satellites; based on a free service; works better in open places

Speedometer

Measures speed using wheel rotations.

Accelerometer

Measures acceleration forces along the three axis

Electronic wheel block

Aliases: electric break, electric block
If turned on, the wheel becomes very hard to rotate

Headlight

Light source embedded in the front to enable night usage

Manual Breaks

Aliases: Breaks, default breaks, disk breaks, rear break
Friction based stopping system; Placed on the back wheel; Controlled by a hand lever.

Accelerator

Handle that controls the speed; If the handle isn't turned the electric break is activated.

Speaker

A device that can play sounds and can be remotely activated.

Mode

Aliases: State
One of the possible states in which the scooter can be. In each mode, the scooter has a different behavior.

Active mode

A mode in which the scooter has no electric limit on power.

Power Saving mode

A mode in which the scooter uses a lower maximum speed to limit power consuption. This happens after the battery gets below a threshold

Power Saving threshold

The battery level below which the scooter enters Power Saving mode.

Standby mode

A mode in which the scooter must be disabled due to extremely low battery.

Standby threshold

The battery level below which the scooter enters Standby mode.

Operations

Unlock

To turn off the electric block

Lock

To turn on the electric block.

Ride

Use the scooter to move.

Disable

To deny the possibility that a customer rents a scooter.

Enable

To allow the possibility that a customer rents a scooter if the business policies allow it.

Maintain

Repair any damage and/or recharge the battery.

Rent

As a customer, paying or exploiting a subscription for unlocking an e-scooter. This action lasts until the scooter is locked again (in normal conditions).

Assign to area

An e-scooter can be assigned to an area if, from that moment, that e-scooter mutually exclusively belongs to that area.

Recover

See recovery.

Ring

Make the speaker play a ringtone, useful to find it.

Relationships

Belong to area

The e-scooter can be only rented and ridden inside that area.

Customer

Alias: user. A customer for the e-scooter service.

Anagraphic information

Collection of data about the customer, including first name, date of birth, and all the other possible information useful for the e-scooter service.

Subscription

A customer has it if they are able to rent e-scooters without paying upfront.

Admin

Alias: manager Works for the E-Scooter Service. Monitors and manages scooters from a control dashboard.

Recovery

The act of picking up an e-scooter, providing maintenance and dropping it somewhere inside the area of service sometime later. The scooter may be unlocked if necessary.

Recovery assistant

Aliases: assistant Works for the e-scooter service and does recoveries with a recovery van.

Recovery van

A vehicle that allows assistants to recover e-scooters.

Map

Bidimensional, polar representation of the Earth surface. Shows, for example, streets, buildings, water courses; what scooters can or cannot cross.

Area of service

Aliases: Service Area The map portion inside which e-scooters assigned to it can be rented.

Drop Point

A place of interest, inside an area of service, where electric scooter are usually dropped after recovery.

Trip

Information about a rent period, during and after its lasting.

Properties

Rider

The rider is the customer that rents the e-scooter during the trip.

Vehicle

The scooter used for transportation.

Distance

The distance, in meters, traveled during the trip.

Start position

Aliases: start
A point inside the area of service where the vehicle was unlocked.

End position

Aliases: end
A point inside the map where the vehicle was locked because the rent ended.

Rent process

See scooter operation rent.

Authorizing Rent

A rent is authorized when the customer has enough virtual currency to pay the unlocking fee.

Rejecting Rent

A rent is rejected when the customer has not enough virtual currency to pay the unlocking fee.

Confirming Rent

A rent is confirmed when it's authorized. This marks the starting time of the rent for time-based charging.

Payment

A payment can be either: - a real-world payment - an in-app payment

Payment legacy system

The preexisting service that manages customer data and payments for the e-scooter service.

Virtual Currency

The amount of imaginary money for every customer. Used to pay scooter rents. It can be obtained through real-world purchases.

Unlocking fee

The price to unlock a scooter before initiating a rent. Charged only once per rent.

Time based charging

The process of charging an amount of virtual currency depending on how much time a rent is taking.

In-App Payment

Aliases: in-app purchase
It's a virtual currency transaction inside the system.

Real-world payment

Aliases: real-world purchase
It's a money transaction from the customer to the business bank account.

Ended: Domain Exploration

Requirement Analysis

Analysis Process

After analyzing the domain, the second meeting with the experts focused on the expected requirements of the whole system.

This interview was conducted following the process of defining user-stories from different perspectives in order to deduce the overall behaviour of the system from the expectations that the customer/experts had about the required functionalities.

A Use Case UML diagram was produced from those stories, detailing how the different use cases were related to each other and to the actors of the system.

In the end, three main actors of the system were identified:

• Customers are the people that rent scooters through the service
• Managers monitor the scooters on a remote dashboard and can inspect the insights to see the suggested drop points
• Assistants are on the field and recover scooters that need to be recharged and drop charged scooters in the suggested drop points

Most of the use cases relate to the Customers and to the processes that need to happen before and while a customer rents a scooter. This is because the expected behavior from the customers point of view is the focus for the business success.

Result

User stories

Some user stories are taken from end users and some other are taken from the business representative, business employees.

1a - Search of a scooter

As a customer, I want to open the mobile application and search for free scooters in my area.

1b - Choose and find a scooter

As a customer, I want to be able to select a scooter near myself and make it ring in order to find it if it's hidden.

1c - Rent a scooter

As a customer, I want to confirm on my phone and start riding. When I'm done, I want to leave the scooter where I can and confirm the end of the trip on the app.

2 - Battery level

As a customer, when I'm searching a scooter, I want to know its battery level in advance.

3a - Area of service policy

When a scooter is taken out of the area of service borders, it locks.

3b - Customer notification of the area border

As a customer, when I'm approaching the area of service border, I want to be warned about the possibility that my scooter gets locked.

How do you want to get warned?

I'd like the scooter to play a specific sound, while I get a notification on my phone.

4 - Power save policy

When the battery goes below a threshold, the scooter enters a power save mode, during which it runs slower than normal.

5 - Customer notification of battery usage

As a customer, I want to be warned when the battery is running low. I want to be warned with a ringtone.

6 - Battery exhausted policy

When the battery is running low, the scooter warns the customer, if present, that the run must end and enters a standby mode during which it's disabled.

7 - Monitoring of scooters

As the business manager, I want to be able to see the position and the battery level of every scooter in a map.

8 - Insights from the system

As the business manager, I want to get suggestions about strategic places for the drop points of the next day.

9 - Unlock policy for employees.

As a business employee I want to be able to unlock freely the scooters in order to move them.

10 - Vandalism prevention

The scooter should discourage theft and vandalism by playing a message when moved while locked or disabled.

11 - Integration with legacy payment system

As the business manager, I want the customers to pay for rides through the existing system.

Does the legacy system keep track of all customers data?

If I remember correctly, yes.

From the legacy system documentation: The legacy payment system needs access to trip data and to customers rent/unlock requests.

Stories diagrams

Renting process diagram

Ended: Requirement Analysis

Domain Analysis

Bounded Contexts

After understanding the domain with the domain experts the team proceeded with the identification of the different subdomains and the bounded contexts within it.

The analysis brought to the definition of the following contexts that were later analyzed to generate the context map in order to understand the relationships between each context and move towards a definition of the different software components that define the architecture of the whole system.

E-Scooter Subdomain

Core Subdomain

Scooter Control & Monitor Context

One of the core contexts. Responsible for keeping track of scooter positions, battery level and other useful data coming from the physical devices. Provides the operation of locking and unlocking scooters to other contexts and it's considered the ground truth on the scooter settings/control data. Also responsible for defining control policies and apply them by operating on the physical devices, like power saving, and enforcing speed limits that could depend on the scooter physical state. Responsible for physical actuation of all the remote operations on the scooter.

Area of Service Context

Keeps track of service areas, scooter-to-area bindings and scooter positions in order to detect when a scooter is crossing the area boundary and notify the other contexts when that happens.

Scooter Data Context

Storage of e-scooters technical and logical static information, like ID, serial number, dimensions, weight... Is the ground truth on the identity of a scooter.

Rent Subdomain

Core Subdomain

Rent Context

Manages the renting operations including storing data on which scooters are rented and by whom. Provides the interface for searching and renting scooters. Manages the logic enabling and disabling of a scooter and provides an interface to do that.

Trip Context

Data collection about trips done by all customers such as starting points, duration, travelled kilometers.

Rent Payment Context

Responsible for managing the payment policy for customer rents. Authorizes the renting operation by checking in with the payment context.

Insight Subdomain

Core Subdomain

Drop points Planning Context

Exploits usage data from other contexts to compute drop point locations or suggestions. Stores data of scooter searches from users.

Payment Subdomain

Supporting Subdomain

Payment Context

Manages all customers in-app transactions, virtual currency, and triggers real-world transactions.

User Subdomain

Generic Subdomain

Customer Context

Keeps track of customers anagraphic data.

Authentication Context

Manages login data and role based authentication for all people that use the service, including admins, assistants and customers.

Context Mapping

The context map below shows all the different context in the scooter service domain.

Five subdomain are identified:

• E-Scooter Subdomain: manages all the logic concerning the physical scooters
• Rent Subdomain: manages all the logic concerning the rent process from the financial policies to the storage of trips' data
• Insight Subdomain: concerns the elaboration of the best possible drop points for scooters to improve the quality of service
• Payment Subdomain: manages the logic of real-world payments by integrating with a legacy system
• User Subdomain: contains the anagraphical information of the service users and provide authentication

The relationships between contexts are explicitated through the representation of interfaces that are exposed in to allow the sharing of information in the system.

Basically all the relationships between services are of type conformist since the context which exposes the interface holds the truth source about that data and it's responsible to decide what to share externally and how.

The payment context is an anti-corruption layer on top of the existing legacy system that manages real world transactions.

Context Map

Domain Models

Common

Common Types

This section contains a set of value objects used throughout the domain models of the identified bounded contexts, to avoid repeating them in each diagram.

Result

Represents the outcome of a domain operation, which may result in a success (in which case the result contains a value of type T) or in a failure (which results in a DomainError).

EntityId

Represents how the identity of entities is passed in the system enabling the possibility to distinguish between one another of the same kind.

GeoPoint

Defines a geographical coordinate as a 2D value (using latitude and longitude).

Constraints:

• \(-90 < latitude.value < 90\)
• \(-180 < longitude.value < 180\)

Distance

Represents a linear, non-directed (meaning an absolute value) distance.

Constraints:

• \(kilometers \geq 0\)

Timestamp

Represents a unique point in time, independent of any time zone.

value is the number of milliseconds passed since 01/01/1970 00:00:00 (Epoch).

Nothing

Represents the absence of any value, nothingness.

Ended: Common

E scooter

Area of Service Domain Model

Class Diagram

Domain Events

• AreaCreated: emitted when an area is created
• AreaDeleted: emitted when an area is deleted
• ScooterAssignedToArea: emitted when a scooter is set to belong to a certain area of service.
• ScooterRemovedFromArea: emitted when a scooter is set to not belong to any area.
• AreaShapeChanged: emitted when the shape of an area of service is changed.
• ScooterWentOutsideArea: emitted when a scooter that belongs to an area is moved outside of its shape.
• ScooterWentInsideArea: emitted when a scooter returns inside its area of service after having moved out.

Scooter Control & Monitor Domain Model

Class Diagram

Details

Duration

Constraints:

• \(milliseconds > 0\)

Speed

Constraints:

• \(metersPerSecond = kilometersPerHour / 3.6\)
• \(metersPerSecond \geq 0\)

BatteryLevel

Constraints:

• \(0 \leq percentage \leq 100\)

Note

If the scooter goes under standby, speed limits imposed by Scooter Control are ignored.

Rules

A scooter has some properties that an employee can set to alter its behavior:

• desiredMaxSpeed: the maximum speed a scooter can reach at any moment;
• powerSavingMaxSpeed: the maximum speed a scooter can reach while in power saving mode;
• powerSavingThreshold: the battery level below which the scooter goes in power saving mode.

The policy of the scooters dictate that, at any moment, the speed of a scooter should be below the desired max speed. Also, while in power saving mode, the scooter must also obey to the powerSavingMaxSpeed rule. Therefore, at any moment the real maximum speed can be calculated as:

• desiredMaxSpeed, if the scooter is in active mode.
• min(desiredMaxSpeed, powerSavingMaxSpeed), otherwise.

Furthermore, a scooter can at any moment go in standby mode. This behavior is dictated by the hardware mounted on the scooter itself. The system reacts to the switch by notyfing all services so that polices can be applied.

Domain Events

• ScooterStatusChanged: emitted after scooter status updates (namely at least one of updateFrequency, locked, maxSpeed or standby).
• TelemetryUpdate: emitted periodically (and frequently) with scooter telemetry updates (namely the current value of the position, battery and speed properties).

Scooter Data Domain Model

Class Diagram

Details

Weight

Constraints:

• \(kilograms > 0\)

Name

Constraints:

• value must not be empty

SerialNumber

Constraints

• value matches ^[A-Za-z0-9-]+$

Domain events

• ScooterCreated: emitted when a new scooter is registered to the system.
• ScooterDeleted: emitted when a scooter is removed from the system.

Ended: E scooter

Insight

Drop Points Planning Domain

Class Diagram

Details

Drop Point

A drop point is a GeoPoint representing a suggested spot where to place some scooters.

Ended: Insight

Rent

Rent Payment Domain Model

Class Diagram

Domain Events

• RentPaymentAuthorized: when a customer has enough credit to pay the unlocking fee.
• RentPaymentRejected: when a customer does not have enough credit to pay the unlocking fee.
• CreditExhaustedForRent: when the credit of a customer is not enough to pay the upkeep cost while riding a scooter for a rent.

Rent Domain Model

Class Diagram

Rules

A scooter has three properties that decide if customers have the ability to rent it.

• enabled: a scooter disabled by an employee cannot be rented;
• standby: a scooter in standby mode cannot be rented;
• outOfService: a scooter outside of its area of service cannot be rented.

While this context has the authority over the enabled property, the remaining are owned by different contexts which Rent has to integrate with.

The combination of the aforementioned properties defines the rentability of a scooter for what concerns the rent context. In addition, a scooter cannot be rented by any customer if another customer is already riding it. This defines its availability state. In order to succeed, a rent request must be made on a scooter that is both rentable and available at the time of the request.

Finally, a constraint exists to prevent customers from renting scooter while another rent is ongoing for them.

Rent lifecycle diagram

Rent Process Diagram

Domain Events

Rent aggregate

• RentRequested: When a customer requests a rent.
• RentConfirmed: When a rent is confirmed.
• RentStopped: When a rent is stopped.
• RentCancelled: When a rent is cancelled.
• RentEnded: When a rent is ended either by cancellation or by stop.

Scooter aggregate

• ScooterEnabled: When a scooter is enabled.
• ScooterDisabled: When a scooter is disabled.
• ScooterBecameRentable: When the combination of enabled/out of service/standby is changed to make the scooter rentable.
• ScooterBecameNotRentable: When the combination of enabled/out of service/standby is changed to make the scooter not rentable.

Trip Domain Model

Class Diagram

Domain Events

• TripStarted: when a new trip starts.
• TripEnded: when a trip ends.

Ended: Rent

Ended: Domain Models

Ended: Domain Analysis

Architecture Design

Architectural map

This section contains an overview of the final version of the system as it has been designed for implementation, giving an holistic view of the developed components and their relationships with each other.

Components of the system can be categorized in:

• Microservices: standalone services that encapsulate their state and provide a well defined set of functionalities accessible through their public interface. Microservices can communicate with each other through events or asynchronous commands.
• Azure services: instances of services offered by the Microsoft Azure cloud infrastructure, providing a wide range of features that need to be integrated with other services.
• Functions: stateless instances that are typically used to translate and facilitate communication among other components of the system. They fit particularly well in the serverless computation model, and are therefore chosen to be implemented through Azure Functions. An important thing to note is that in Azure, the minimal unit of deployment is a Function App, which can host multiple functions with different tasks that scale together. The following sections use the term "function" to indicate an Azure Function App, that can therefore contain multiple instances of serverless functions.
• Devices: computational resources belonging to the physical world. They are embedded IoT devices onto the electric scooters managed by the system.

The diagrams in the sections below show the communication between each pair of components, classifying it in:

• Synchronous updates (Updates arrow): commands sent over synchronous media, like HTTP or RPC.
• Asynchronous commands (Sends commands arrow): commands sent in an asynchronous way, typically using message queues.
• Publish subscribe (Exposes topic & Observes arrows): events are published on topics and observed by subscribers of that topic, using asynchronous messaging.

E-Scooter subdomain

This subdomain contains all the services and functions that deal with the scooter lifecycle and with the exchange of information between the physical devices and the system.

Since the scooter data context has the authority over the identity of the scooters and, thus, over their existence in the system, we planned to build it as a microservice. However, for this prototypal implementation, due to the limited time, it was mocked via a GUI that mimics its behavior exposing the scooter lifecycle interface (i.e. notifying the creation/deletion of scooters).

On the other hand, the Scooter Monitor & Control context deals with the communication with the physical world. Its access point to the real world has been chosen to be realized via the Azure IoT Hub, a service specifically designed to be the pivot point between the cloud and IoT devices.

The IoT Hub can keep track of a large set of entities (denoted as Devices), each with its own identity. Each device has two sets of properties to hold state: (i) Desired properties, used by the cloud infrastructure to communicate the desired state for the device; (ii) Reported properties, used by the device to communicate its actual state (which can be different from the desired one). Furthermore, each device can emit a series of frequent events (almost a continuous stream) denoted as Telemetry. These are meant for continuously changing properties (like position, speed or battery level, in the e-scooter domain).

New scooters that are registered to the system are recorded as devices inside the IoT Hub thanks to the Manage Devices function, relying on the scooter lifecycle topic. After registration, the scooter can start to be monitored and/or controlled by the system.

The physical actuation of control policies is demanded to the Scooter Control function, which receives asynchronous commands from other components in order for them to be propagated to the physical devices via the IoT Hub. This commands, among others, include the operations to lock/unlock scooters, for example when a customer rents one.

The IoT Hub can also be configured to emit events whenever a telemetry is sent or whenever a reported property changes. This feature has been exploited to integrate it with the rest of the ecosystem. In particular, the cloud service emits events when properties like locked or standby change. In order to have a simpler interface for reported properties updates, the Scooter Monitor function has been put in between the Hub and the rest of the system, to translate them into events tailored for the e-scooter domain.

User subdomain

Being a generic subdomain, the team decided to have the User subdomain be implemented using an off-the-shelf package. Therefore, the team opted to mock the interfaces of this subdomain that were needed by other components in other subdomains, namely the Customer lifecycle topic. This is the topic exposed by the designed Customer Microservice to let others know about customers signing up to the system or customers unregistering from it. Like the scooter data microservice, this was implemented via a GUI emitting the events for other services to observe.

Rent subdomain

The rent subdomain deals with the functionalities that allow customers to rent scooters, along with side features reserved to assistants and admins of the service. The core of this subdomain is the Rent Microservice, which provides clients with an interface to start and stop rents as a customer, or to enable or disable scooters as an assistant or as an admin. To work properly, it needs to know about the existing customers and scooters, which is done by subscribing to the Scooter Lifecycle, Scooter Status and Customer Lifecycle topics. This service is also responsible for the locking and unlocking of scooters according to the lifecycle of the related rents. Since these operations are offered by the Scooter Control function, the rent service only needs to send asynchronous commands to it when necessary.

The Rent Microservice does not directly deal with the payment of the rides, but delegates this responsibility to another microservice from a different bounded context, the Rent Payment Service. Since payments are assumed to use virtual currency managed by the system, this service needs to ensure that: (i) a renting customer has enough credit to start a rent, that is authorizing or rejecting the rent; (ii) a rent is stopped when a customer's credit runs out. This collaboration between the two services is carried out via an exchange of events using the Rent Lifecycle and Rent Payment Events topics. Due to the limited time though, the Rent Payment service was mocked with a function that always authorizes rents when they are requested.

Azure Digital Twins

Since this project is about the exploration of the digital twins paradigm, the team decided to adopt a standard and production-ready solution to keep track of the real-time state of the system. This gives clients the possibility to query it in arbitrary ways. The solution of choice was Azure Digital Twins, which has all the required features.

Azure Digital Twins allows developers to define a set of DTDL models to define real-world assets as twins along with their properties and the relationships they can have with other digital twins. Once the models are uploaded to the Azure Digital Twins instance, external agents can use its API and/or SDK to run CRUD operations on both digital twins and relationships, in order to control the digital twin graph.

EScooter DTDL Models

DTDL models of EScooter digital twins can be found in the twins-models repository.

Azure Digital Twins can be easily integrated with other cloud services (including the IoT Hub) using Azure Functions. In particular, the team planned to develop a group of functions to create digital twins for the scooters and the customers managed by the rest of the system and also to keep them up to date with the real-time state.

Service Architecture

Clean architecture

Each service of the system follows a strict architecture that splits it into a series of layers, following the principles of the Clean Architecture. Since there are several different opinions and views on what the clean architecture should look like, the team came up with its own interpretation of the layers contained in each service and of their relationships with each other. While the details of each layer are discussed in the next sections, the diagram below shows a holistic view the resulting design.

Domain Layer

This is the core layer of any service and it is responsible of defining the business rules of the context containing the service. It contains the definition of the entities and value objects belonging to the service and their associated behaviors. Entities and value objects are then organized into aggregates enforcing consistency boundaries over them. This layer can also define high level processes or constraints that are encapsulated inside domain services. Entities and domain services can notify the occurrence of any relevant state change using domain events, which are defined here but handled within the application layer.

Application Layer

Defines a collection of use cases exposed by the service and the actions to be executed for each of those through a series of handlers that interact with the domain and infrastructure layers to carry out those actions. Handlers can also be used to run business logic when events occur in the domain or to integrate with other services/contexts using external events. External events are a projection of domain events that is propagated to other contexts, useful to hide inner domain details, therefore making contexts less coupled. Furthermore, this layer is responsible of authorization, transactional data access and communication with external services. To keep the application logic as clean as possible, infrastructural concerns like databases, event queues or protocols are kept away from the application layer through infrastructure interfaces declared here, but implemented by the infrastructure layer itself.

Infrastructure Layer

This layer holds the concrete implementation of the interfaces declared by the Application Layer or by the Domain Layer. These interfaces typically model behavior related to persistence, external communication and other utility services required by other layers.

Web Layer

Implements the entry point that clients of the service use to make requests. Clients can make requests using the REST API exposed by each microservice, therefore this layer defines the REST endpoints that the service supports along with their interface. To that purpose, each service contains a series of DTOs (Data Transfer Objects, i.e. the objects modeling the format of communication with clients) to formally represent the interface of each endpoint. Lastly, this layer is responsible of managing the authentication process to determine the identity of the agent making requests to the service.

Ended: Architecture Design

DevOps Engineering

DevOps is considered a fundamental part of a microservice based system. This section will be discuss the tools and methodologies put in practice by the team.

DVCS Strategy

Having a large set of repositories, each typically maintained by a single developer (or at most two using pair-programming), the chosen branching strategy is Trunk-based. This simplifies the management of the repositories by individual developers, while still giving them the possibility to occasionally open feature/hotfix branches if needed.

Licenses

Each piece of software made is provided under the MIT License, which makes it FOSS.

Versioning

Each of the softwares developed is versioned with the Semantic Versioning system. Git annotated tags were used to declare new versions and trigger the release system. A tool was implemented to search for the latest tag and generate a version for the build automation.

Software Build Lifecycle

Every step of the software build lifecycle was automated through Git and GitHub Actions. GitHub Actions offers hosted containers (called runners) that run many kind of scripts divided in flows and tasks, in an imperative fashion.

Many script reuse techniques are possible. The team employed:

• Custom Actions
• Reusable Workflows
• Repository templating

Principles adopted

These are the principles adopted for every repository:

• Commits should adhere at the Imperative tense, impersonal form.
• Versions are managed with git tags, specifically in the form v#.#.#.
• At each push, software build and tests are performed.
• When a new tag is pushed, after a successful test job, a release is performed.
• Releases happen on the GitHub Release page and trigger a deploy.
• Azure Cloud was the deploy destination for each software components, so that part of the workflow is shared between similar projects.
• Consistency between .NET compilation target, runtime target, build-time dotnet version.

GitHub Actions Workflows

Workflows define the automation processes of a repository. An example of a workflow that was created by the team and was used in this project can be found here.

Workflows are composed by jobs which can run concurrently or in sequence based on a graph of dependencies between each other. Each job is composed by a sequence of steps to be made. Steps can fail. When it happens, normally the job is interrupted and flagged as failed. Steps can run scripts or actions.

Every action takes care of one build step. Some custom actions aggregate repeated steps into a single one (see Composite Actions).

Actions the team developed in first person

Some members of the team had previously developed custom GitHub Actions for their projects and reused them for EScooter. These are:

• Markdown Docs - Generates a website from markdown files.
• SemVer Checkout - Checkouts the full history of a repo to determine the software version with semantic versioning.
• SemVer Release - Creates a GitHub release of the software artifacts using its semantic version, generating a changelog.
• Dotnet Build - Runs dotnet restore & dotnet build.
• Dotnet Test - Runs the tests with dotnet.
• Dotnet Publish - Produces a folder with the final .exe file and all its dependencies.

Reusable workflows

After the release of reusable workflows by GitHub, DRY principles were applied extensively on CI/CD code too. Two unique workflow file were created so that each piece of software implemented with .NET would reference them instead of repeating the same code and configuration. One was created for Azure Functions CI/CD to include delivery on the cloud resource, one for Dekstop applications and Hosted Runners CI/CD.

Workflow Design

The two main workflows the team designed are:

• Azure Functions CI
• C# App CI

They both start with a build job (Continuous Integration) consisting of checkout, building (compiling + linking), unit testing, publishing (artifact generation) and artifact uploading (artifacts on GitHub aren't volatile, they are kept for about a day based on the configuration).

The release (Continuous Delivery) job is run only if a new tag respecting Semantic Versioning gets pushed. This job creates a GitHub release and uploads the binaries.

The Azure Functions CI provides also the deploy job (Continuous Deployment), which uploads the function to the relative slot on Azure Cloud.

The analyze job, present in both the workflows, builds the sources with CodeQL and runs all the default code quality and security queries, provided by the CodeQL team, reporting results in the Security tab of the repository. The outcome of the analysis is ignored for the purposes of build success, because CodeQL proved to be slightly unstable and too computational expensive (time and memory) to be waited every time.

Other reusable workflows adopted

Microservices CI/CD uses the following workflows:

• Microservice Build
• Microservice Azure Deploy

These workflows were developed by some members of the team for a different project, and reused within the Rent service repository.

Repository templates

Repository templates allow a faster bootstrap phase of a project, scaffolding a hello world version of the program to be developed. This includes the automatic generation of the build lifecycle from the very first commit.

The template used for every EScooter Azure Function can be found here.

Microservice template

A template used for microservices but made for a different project can be found here. It makes use of the Clean Architecture library developed by some members of the team for every microservice based on the Clean Architecture and .NET.

Other tools

A script was created for the Git Bash that adds a git release command. The purpose is to quickly create the new patch, new minor or new major version of some software based on the repository history and semantic versioning. The code can be found here.

Quality control

Every C# project was configured with a consistent style rulesheet inside the .editorconfig file. StyleCop Analyzers takes care of enforncing the style designed both during development and during automated builds.

Check

Style warnings are treated as errors.

Static Code Analysis

Static code analysis for the C# language is performed automatically within the CI pipeline for every Azure function project. Reports are put under the Security tab of the relative GitHub project. The tool used for the analysis is CodeQL. Both security and code quality alerts are enabled.

Info

CodeQL security and code quality warnings and errors are ignored by the workflows and are simply treated as suggestions. The reason is the instability of the CodeQL suite (bot the software and the default ruleset) which isn't always updated/fixed in time.

Documentation Tools

Automated documentation generation is provided of both the overall design of the system and the API of every microservice. Microservices use a Swagger plugin for ASP.NET (see the Rent Service API docs as an example).

Ignored aspects

Public repository (NuGet)

Given the fact that the EScooter project didn't include libraries, but only single-purpose services and functions, NuGet wasn't used.

Dependency inspection/automatic management

The team voluntarily ignored any real-time dependency and vulnerability management because of the finite scope of the project. GitHub Dependabot creates automatic pull requests that suggest dependency updates. The team wanted to avoid receiving those PRs forever.

Implementation

Implementation

This section explains how the different parts of the system are organized and implemented.

The choice was to carry out the full analysis of a complex domain to be able to approach a "real-world" problem in its entirety and use the Domain Driven Design techniques to study it, but as far as the implementation goes, we always knew that it was not needed to implement the whole system, so we decided to focus on the core and interesting aspects, mock the supporting services and completly ignore some of the features that were outside of this core prototype boundary scope.

In the end, the most interesting part was the implementation of the Scooter Monitor & Control context in order to realize the Digital Twin layer that was at the core of the Pervasive Computer course for which this project will be submitted too. This meant that the Scooter Data context was mocked since it was just a collection of data relevant for the domain but not too much for the functionalities of the system.

Renting operations in the Rent Context were implemented to have at least one standalone microservice in place to challenge ourselves with the requirements of the clean architecture. The implementation of the Area of Service context was put aside in order to focus on the most relevant use-case of the system itself which was the scooter rent management. In an iterative development process of the real system this would have probably been the smartest choice to, at least, guarantee the basic functionalities immediatly and later add the other planned features.

With the same reasoning the implementation of the Insights subdomain and the connected Trip Context were skipped and the Rent Payment context was mocked since we didn't need to manage the virtual currency and the relationship with the Payment subdomain.

The User subdomain was plain both from a design and implementative perspective so the group didn't focus too much on that. Of course in a real-world scenario authorization and security should have been put in place since the very beginning, but for a proof-of-concept like the one that was developed this was definetly not needed nor particularly interesting.

Scooter Service Backend

The backend is fully deployed on Microsoft Azure: each resource is under the e-scooter resource group.

The next sections describe the different requirements of the system and how they were realized using resources deployed on the Azure Cloud.

Communication with the Physical devices

The connection with the physical devices is managed through an IoT Hub named scooter-iot-hub. The IoT Hub manages devices representing them with a set of desired and reported properties and some metadata describing the device itself. Services can edit desired properties of the digital device which are pushed as events on the physical device itself and can trigger changes in the device status. Notifications on status changes can be achieved either from the use of reported properties when changes are not frequent or through telemetry events that are usually proactive streams of data coming from the physical device's sensors.

Data aggregation

The layer that aggregates the data from all the different sections of the system is implemented making use of Azure Digital Twins and is named scooter-digital-twins. This layer stores the latest updated state of the whole system keeping data about customers, scooters and rents in an eventually consistent source. This means that the Digital Twin layer offers a comprehensive view of the domain to query the state of the system instead of interrogating each microservice separately.

The result structure is a Digital Twin Graph as provided by the Azure tool that allow to store current state of the twins and relationships between each other. Here are the models used on the graph for Scooters and Customers twins which are the main entities modelled.

Inter-Service communication

Communication among services is carried out in an asynchronous fashion using events queues.

• Events are carried around between services using a Service Bus named scooter-event-bus on the topic production/service-events.
• Each subscriber to the scooter-event-bus manages its own subscription filtering only the relevat events to receive.
• Telemetry events from the devices is transmitted through an Event Grid from the IoTHub.

Event Handling

Azure functions are generally used as event handlers to map the communication from a service to another and especially when dealing with updating the Digital Twin Layer.

Here is a list of all the implemented Azure functions:

• manage devices is a function that manages the creation and removal of scooters on the IoTHub and on the Digital Twin Layer.
• manage customer is a function that manages the creation and removal of the customers' digital twins.
• manage rents is a function that manages the creation and removal of renting relationships on the digital twin graph.
• manage telemetry updates updates the state of each scooter digital twin when new telemetry is received by the IoTHub (e.g. position, speed).
• manage reported properties updates the state of each scooter digital twin when reported properties are updated on the device (e.g. lock state, standby).
• manage scooter availability update the state of each scooter digital twin when the availability is changed from an administrator of the system on the rent service.

Two functions instead of managing the state of the Digital Twin Graph are extending the functionalities of the IoTHub to implement the Scooter Control & Monitor Context:

• scooter control is a function that exposes the default applicable actions on the devices: in particular, it manages the lock/unlock command and listens to the telemetries from the IoTHub to apply domain policies (e.g. modifying max speed according to battery level).
• scooter monitor is a function that exposes the events concerning changes of the device state (e.g. lock, standby)

Functions were also used to implement mocked parts of the system when needed. For example the rent payment which is a mock implementation of the Rent Payment context that always confirm the rents.

Microservices

Microservices are implemented with an App Service. Only the Rent service was developed as a fully standalone microservice that maps the Rent Context.

Frontend

Admin frontend

The Admin frontend is implemented using Angular: polling the Digital Twin Layer, it shows the scooters in a 2D map with an icon colored depending on the state of each scooter:

• Green when it's free;
• Red when it's currently rented;
• Yellow when it is in standby;
• Grey when it is disabled.

By clicking on a scooter it is possible to check the value of its properties and enable or disable it to allow or prevent renting.

Customer frontend

The Customer frontend is implemented in C# using WPF: it is a very simple mock of the mobile app that users could use to rent an available scooter.

Scooter Data mock

The Scooter Data Mock is a mock implementation with WPF of the Scooter Data context. It allows to create a new scooter identity or to remove an existing one by generating the corresponding events which are then handled by the rest of the system accordingly. It does not store any data about the scooter since this was not relevant for the other core services.

Customer mock

The Customer Mock is a mock implementation with WPF of the Customer context. It allows to create a new customer identity or to remove an existing one by generating the corresponding events which are then handled by the rest of the system accordingly. It does not store any data about the customer since this was not relevant for the other core services.

Device emulator

The Device emulator simulates the typical behavior of e-scooters in the real world, allowing us to live-test the entire system.

Implemented as HostedService with C# .NET, the emulator executes a loop in which queries for new or updated scooters and simulates a random realistic usage to update them: it moves the scooters, uses the battery and sets standby mode eventually, as if humans were using them to move around Cesena.

The requirement analysis of the device emulator can be found here.

Device Emulator

Device Emulator Requirements

1. Emulate all the devices on a Iot Hub instance

   1. Fetch the list of all devices, periodically
   2. For each device, start an infinite emulation that lasts until a "stop event" is fired
      1. A stop event can be a keyboard interrupt or a C# method call
      2. The emulation consists of:
         1. Load device data
         2. Decide which property should be changed, why and how
            1. If the device is locked, do nothing
            2. If it's unlocked, randomly select an action:
               1. Do nothing
               2. Change position based on velocity (only if velocity is not 0)
               3. Change position and change velocity
            3. If it's moving, consume some battery
               1. If the battery is below the standby threshold go in standby mode
         3. Change the property values
         4. Submit the update to IoT Hub

   Note

   Emulating a device means firing a sequence of realistic updates on its reported properties on IoT Hub. We consider every device to be an E-Scooter.

2. Can be executed both as script and as a library

Ended: Device Emulator

Digital Twins

Available also in the repository.

Scooter DT

Model

{
  "@context": "dtmi:dtdl:context;2",
  "@id": "dtmi:com:escooter:EScooter;1",
  "@type": "Interface",
  "displayName": "E-Scooter",
  "contents": [
    {
      "@type": ["Property", "TimeSpan"],
      "name": "UpdateFrequency",
      "schema": "integer",
      "unit": "second",
      "writable": true
    },
    {
      "@type": "Property",
      "name": "Locked",
      "schema": "boolean",
      "writable": true
    },
    {
      "@type": "Property",
      "name": "Enabled",
      "schema": "boolean",
      "writable": true
    },
    {
      "@type": ["Property", "Velocity"],
      "name": "MaxSpeed",
      "schema": "double",
      "unit": "kilometrePerHour",
      "writable": true
    },
    {
      "@type": "Property",
      "name": "Connected",
      "schema": "boolean"
    },
    {
      "@type": "Property", 
      "name": "Standby",
      "schema": "boolean"
    },
    {
      "@type": "Property",
      "name": "BatteryLevel",
      "schema": "double",
      "comment": "percentage semantic type is missing"
    },
    {
      "@type": ["Property","Latitude"],
      "name": "Latitude",
      "schema": "double",
      "unit": "degreeOfArc"
    },
    {
      "@type": ["Property","Longitude"],
      "name": "Longitude",
      "schema": "double",
      "unit": "degreeOfArc"
    },
    {
      "@type": ["Property","Velocity"],
      "name": "Speed",
      "schema": "double",
      "unit": "kilometrePerHour"
    }
  ]
}

Customer DT

Model

{
  "@context": "dtmi:dtdl:context;2",
  "@id": "dtmi:com:escooter:Customer;1",
  "@type": "Interface",
  "displayName": "Customer",
  "contents": [
    {
      "@type": "Relationship",
      "name": "is_riding",
      "displayName": "is riding",
      "target": "dtmi:com:escooter:EScooter;1",
      "properties": [
        {
          "@type": "Property",
          "name": "start",
          "schema": "dateTime"
        }
      ]

    }
  ]
}

Deployment Diagram

Ended: Implementation

Conclusions

This project proved to be a great challenge in trying to be coherent through time with all the notions from the course.

Keeping a clean developement process during the timespan of the project was quite hard partially due to both the use of the Azure Cloud, which was new for basically all the team members, and the fragmentation in time dedicated to the project caused by other personal and academic obligations that didn't allow to have a smooth route from start to finish.

Even so, the team agrees that it was a good way to see in practice the strenghts and challenges of the techniques studied during the course in a "real world" scenario.

Domain Driven Design

Concerning the Domain Driven Design the team started with a long planning session to try to pin down all the concepts and have a clear holistic view of the whole system.

Then after studying the technology chosen to implement the system itself some clashes emerged and it was a bit harder to constantly update the documentation to match in real-time what was happening with the software.

This was identified as the most challenging aspect of the DDD, whilst the analysis process carried out with the tools and the philosophy transmitted during the course felt quite natural and effective since the beginning.

In retrospective the team feels that some cleaner and more coherent with the Ubiquitous Language code could have been created, but, while developing, that felt unnecessary and overcomplex for smaller pieces of software like the Azure Functions and instead was useful for bigger artifacts like the services and the scooter-emulator.

DevOps process

From a DevOps perspective the team put a lot of effort in the automatization of the quality assurance and the deployment of the whole system. This really helped in having a clean, self updating system especially since the final product was a cloud service.

Testing was a bit left behind primarily for time management reasons and because most of the testing was done while integrating different software components instead of having complex business logic that needed intensive single unit tests.

Overall setting up a pipeline from the very beginning resulted in a helpful tool that guided each team member during the development of the system.