As we have discussed already, for each data type, there are also other factors that will determine which storage mechanism we choose.
We need to think about how we will receive the data, how we will interact with the data, and how we will present it back to the user.
User-interactive applications
If data will be used as a backend to a business application, such as an e-commerce site, then our data is typically product information, customer records, and order records. This data is highly structured and needs to be quickly queried, and be managed through a user interface (for example, the web page). You might also want to batch import data or store reports, but these are secondary mechanisms, as they are not what the application does minute by minute or day to day.
A database is an obvious choice; however, it might not be the only mechanism. An e-commerce site will usually store and display product images. Although it is possible to store images directly inside a database, this is often not the best choice as additional computing will be required to convert the data from a database field type into an image type.
In this scenario, we can consider using a hybrid of a database and a traditional file store. A field in a database’s product table will hold a reference to the image (that is, its filename), but the image itself could be stored and retrieved from Azure Blob Storage.
Other media files or document file types can also be used in a similar way. For example, a database record holds a reference, but the actual file is stored or retrieved from a file-based storage container, such as Azure Blob Storage or Azure File Storage:
Figure 9.2 – An example of a user-interactive application
As well as web-based interactive pages, another increasingly common use case for Azure is to analyze big data.
Data analytics solutions
Another common scenario is a data analysis application. Here, data might be received in bulk as structured or semi-structured files. These files need to be read, queried, and maybe even reformatted to produce a report or data model.
The initial storage mechanism could be a file store such as Azure Blob Storage or Azure Data Lake. From there, you could ingest the data into a database store such as Azure SQL or Cosmos DB. Again, we use a mixture of technologies, but each has its specific use case – for instance, is it being used purely for storage or for querying? Take a look at the following diagram:
Figure 9.3 – An example of a data analytics solution
Most solutions will also need to store logs. Either for troubleshooting or performance monitoring, logging needs to be carefully planned.
Leave a Reply