If you upgraded AKS recently (to v1.23+), you might have noticed some containers stopped working, most of the time failing to start with messages like “Error: failed to start containerd task “solr”: hcs::System::CreateProcess solr: The system cannot find the file specified.: unknown“.
As we spent quite some time researching those issues and also contacted Sitecore support, I’ve decided to write this post so it can be helpful to anyone else facing the same kind of issues.
Even if Docker runtime is still available in v1.23, it comes with Containerd by default so you will get that kind of exception. But bare in mind Docker is going to be fully removed on v1.24 so I suggest you take action as soon as possible to avoid blocking upgrading or facing issues later.
About Sitecore default images
In case you were making use of Sitecore images =< v10.0.2 then you will find those failing to start, mostly on the solr-init and mssql-init containers.
Sitecore has fixed the images for the following versions:
10.0.3
10.1.2
10.1.3
10.2.0
10.2.1
Please notice that if you are referring to the image versions using the “two-digit” tag, then you’re good to go, as would be getting its latest version.
About the runtime deprecation
AKS announced the deprecation of Docker in version 1.20:
Dependency on Docker explained
A container runtime is software that can execute the containers that make up a Kubernetes pod. Kubernetes is responsible for orchestration and scheduling of Pods; on each node, the kubelet uses the container runtime interface as an abstraction so that you can use any compatible container runtime.
In its earliest releases, Kubernetes offered compatibility with one container runtime: Docker. Later in the Kubernetes project’s history, cluster operators wanted to adopt additional container runtimes. The CRI was designed to allow this kind of flexibility – and the kubelet began supporting CRI. However, because Docker existed before the CRI specification was invented, the Kubernetes project created an adapter component, dockershim. The dockershim adapter allows the kubelet to interact with Docker as if Docker were a CRI compatible runtime.
Switching to Containerd as a container runtime eliminates the middleman. All the same, containers can be run by container runtimes like Containerd as before. But now, since containers schedule directly with the container runtime, they are not visible to Docker. So any Docker tooling or fancy UI you might have used before to check on these containers is no longer available.
You cannot get container information using docker ps or docker inspect commands. As you cannot list containers, you cannot get logs, stop containers, or execute something inside a container using docker exec.
Please refer to the official documentation for deeper details:
Ok, so now that things got a bit clear, and we know Sitecore base images are fixed in the latest versions (at least for v10), what about our custom ones?
So far, I’ve identified some changes required on our Dockerfile to make it work as expected in Containerd runtime.
ENTRYPOINT and CMD
The syntax is slightly different, I’ll share examples so it’s even easier to understand the changes.
In my previous posts about images cropping, I’ve used Azure Cognitive Services (Vision) for managing media cropping in a smart way. Now, I’m sharing another usage of Azure Cognitive Services (Language) for building a Powershell tool that makes possible to translate your Sitecore content in a quick and easy way.
Handling item versioning and translation from the Sitecore content editor is a kinda tedious work for editors, especially when it comes to manually creating localized content for your site.
The idea of the PSE tool is to make the editor’s life easier, so in several clicks can achieve the language version creation of the item (including subitems and datasources) and also populate the items with translated content!
Azure Translator – An AI service for real-time text translation
Translator is a cloud-based machine translation service you can use to translate text in near real-time through a simple REST API call. The service uses modern neural machine translation technology and offers statistical machine translation technology. Custom Translator is an extension of Translator, which allows you to build neural translation systems. The customized translation system can be used to translate text with Translator or Microsoft Speech Services. For more info please refer to the official documentation.
About the tool
As I mentioned before, this tool is based on SPE, so it’s easy to integrate on your Sitecore instance. I’ll share the full implementation details but also the code and packages. The service API layer has been implemented on .NET.
The context menu script
Demo
Creating the Azure service
Before proceeding with the implementation, let’s see how to create the Translator service in Azure. The steps are very straightforward as usual when creating such resources.
That’s it! You have your translator service created, now just take a look at the keys and endopint section, you will need it for updating in your config file:
Keys and Endopint
Service implementation (C#)
TranslatorService.cs
This is the service that communicates with the Azure API, it’s quite basic and straightforward, you can also find examples and documentation in the official sites.
using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;
using Sitecore.Cognitive.Translator.PSE.Caching;
using Sitecore.Cognitive.Translator.PSE.Models;
using Sitecore.Configuration;
namespace Sitecore.Cognitive.Translator.PSE.Services
{
public class TranslatorService : ITranslatorService
{
private readonly string _cognitiveServicesKey = Settings.GetSetting($"Sitecore.Cognitive.Translator.PSE.TranslateService.ApiKey", "");
private readonly string _cognitiveServicesUrl = Settings.GetSetting($"Sitecore.Cognitive.Translator.PSE.TranslateService.ApiUrl", "");
private readonly string _cognitiveServicesZone = Settings.GetSetting($"Sitecore.Cognitive.Translator.PSE.TranslateService.ApiZone", "");
public async Task<TranslationResult[]> GetTranslatation(string textToTranslate, string fromLang, string targetLanguage, string textType)
{
return await CacheManager.GetCachedObject(textToTranslate + fromLang + targetLanguage + textType, async () =>
{
var route = $"/translate?api-version=3.0&to={targetLanguage}&suggestedFrom=en";
if (!string.IsNullOrEmpty(fromLang))
{
route += $"&from={fromLang}";
}
if (!string.IsNullOrEmpty(textType) && textType.Equals("Rich Text"))
{
route += "&textType=html";
}
var requestUri = _cognitiveServicesUrl + route;
var translationResult = await TranslateText(requestUri, textToTranslate);
return translationResult;
});
}
async Task<TranslationResult[]> TranslateText(string requestUri, string inputText)
{
var body = new object[] { new { Text = inputText } };
var requestBody = JsonConvert.SerializeObject(body);
using (var client = new HttpClient())
using (var request = new HttpRequestMessage())
{
request.Method = HttpMethod.Post;
request.RequestUri = new Uri(requestUri);
request.Content = new StringContent(requestBody, Encoding.UTF8, "application/json");
request.Headers.Add("Ocp-Apim-Subscription-Key", _cognitiveServicesKey);
request.Headers.Add("Ocp-Apim-Subscription-Region", _cognitiveServicesZone);
var response = await client.SendAsync(request).ConfigureAwait(false);
var result = await response.Content.ReadAsStringAsync();
var deserializedOutput = JsonConvert.DeserializeObject<TranslationResult[]>(result);
return deserializedOutput;
}
}
}
}
The code is simple, I’m just adding a caching layer on top to avoid repeated calls to the API.
You can check the full parameters list in the official documentation, but let me just explain the ones I used:
api-version (required): Version of the API requested by the client. Value must be 3.0.
to (required): Specifies the language of the output text. The target language must be one of the supported languages included in the translation scope.
from (optional): Specifies the language of the input text. Find which languages are available to translate from by looking up supported languages using the translation scope. If the from parameter is not specified, automatic language detection is applied to determine the source language.
textType (optional): Defines whether the text being translated is plain text or HTML text. Any HTML needs to be a well-formed, complete element. Possible values are: plain (default) or html. In this case, I’m passing the HTML when is translating from a Rich Text field.
We need also to create the models where the data is parsed into (TranslationResult), I’m not adding the code here to make it simple, but you can check the source code for full details.
TranslationExtensions.cs
using System.Linq;
using System.Threading.Tasks;
using Sitecore.Cognitive.Translator.PSE.Services;
using Microsoft.Extensions.DependencyInjection;
using Sitecore.DependencyInjection;
namespace Sitecore.Cognitive.Translator.PSE.Extensions
{
public class TranslationExtensions
{
private readonly ITranslatorService _translatorService;
public TranslationExtensions(ITranslatorService translatorServices)
{
_translatorService = translatorServices;
}
public TranslationExtensions()
{
_translatorService = ServiceLocator.ServiceProvider.GetService<ITranslatorService>();
}
public async Task<string> TranslateText(string input, string fromLang, string destLang, string textType)
{
var res = await _translatorService.GetTranslatation(input, fromLang, destLang, textType);
if (res != null && res.Any() && res[0].Translations.Any())
{
return res[0].Translations[0].Text;
}
return string.Empty;
}
}
}
We need basically one main script to be added in the context menu (Add Language Version and Translate) and then few functions that has been written in this way to make it more readable and modular.
Add Language Version and Translate
Import-Function GetLanguages
Import-Function GetItems
Import-Function ConfirmationMessage
Import-Function Translate
Import-Function GetUserOptions
Import-Function GetUserFieldsToTranslate
Import-Function ConfirmationMessage
# Global variables
$location = get-location
$currentLanguage = [Sitecore.Context]::Language.Name
$langOptions = @{}
$destinationLanguages = @{}
$options = @{}
# Variables from user input - Custom Object
$userOptions = [PSCustomObject]@{
'FromLanguage' = $currentLanguage
'ToLanguages' = @()
'IncludeSubitems' = $false
'IncludeDatasources' = $false
'IfExists' = "Skip"
'FieldsToTranslate' = @()
}
# Get language options
GetLanguages $langOptions $destinationLanguages
# Ask user for options
$result = GetUserOptions $currentLanguage $langOptions $destinationLanguages $userOptions
if($result -ne "ok") {
Write-Host "Canceling"
Exit
}
# Get all items
$items = @()
$items = GetItems $location $userOptions.IncludeSubitems $userOptions.IncludeDatasources
# Ask user for fields to translate
$dialogResult = GetUserFieldsToTranslate $items $options $userOptions
if($dialogResult -ne "OK") {
Write-Host "Canceling"
Exit
}
# Ask user for confirmation
$proceed = ConfirmationMessage $items.Count $options $userOptions
if ($proceed -ne 'yes') {
Write-Host "Canceling"
Exit
}
# Call the translator service
Translate $items $userOptions
GetLanguages
function GetLanguages {
[CmdletBinding()]
param($langOptions, $destinationOptions)
$user = Get-User -Current
$languages = Get-ChildItem "master:\sitecore\system\Languages"
$currentLanguage = [Sitecore.Context]::Language.Name
# Get list of languages with writting rights and remove the origin language
foreach ($lang in $languages) {
$langOptions[$lang.Name] = $lang.Name
if (Test-ItemAcl -Identity $user -Path $lang.Paths.Path -AccessRight language:write) {
$destinationOptions[$lang.Name] = $lang.Name
}
}
$destinationOptions.Remove($currentLanguage)
}
GetUserOptions
function GetUserOptions {
[CmdletBinding()]
param($currentLanguage, $langOptions, $destinationLanguages, [PSCustomObject]$userOptions)
# Version overwritting options
$ifExistsOpts = @{};
$ifExistsOpts["Append"] = "Append";
$ifExistsOpts["Skip"] = "Skip";
$ifExistsOpts["Overwrite"] = "OverwriteLatest";
$result = Read-Variable -Parameters `
@{ Name = "fLang"; Value=$currentLanguage; Title="From Language"; Options=$langOptions; },
@{ Name = "tLang"; Title="Destination Languages"; Options=$destinationLanguages; Editor="checklist"; },
@{ Name = "iSubitems"; Value=$false; Title="Include Subitems"; Columns = 4;},
@{ Name = "iDatasources"; Value=$false; Title="Include Datasources"; Columns = 4 },
@{ Name = "iExist"; Value="Skip"; Title="If Language Version Exists"; Options=$ifExistsOpts; Tooltip="Append: Create new language version and translate content.<br>" `
+ "Skip: skip it if the target has a language version.<br>Overwrite Latest: overwrite latest language version with translated content."; } `
-Description "Select a the from and target languages with options on how to perform the translation" `
-Title "Add Language and Translate" -Width 650 -Height 660 -OkButtonName "Proceed" -CancelButtonName "Cancel" -ShowHints
$userOptions.FromLanguage = $fLang
$userOptions.ToLanguages += $tLang
$userOptions.IncludeSubitems = $iSubitems
$userOptions.IncludeDatasources = $iDatasources
$userOptions.IfExists = $iExist
return $result
}
GetItems
function GetItems {
[CmdletBinding()]
param($location, $includeSubitems, $includeDatasources)
Import-Function GetItemDatasources
$items = @()
$items += Get-Item $location
# add subitems
if ($includeSubitems) {
$items += Get-ChildItem $location -Recurse
}
# add datasources
if ($includeDatasources) {
Foreach($item in $items) {
$items += GetItemDatasources($item)
}
}
# Remove any duplicates, based on ID
$items = $items | Sort-Object -Property 'ID' -Unique
return $items
}
GetFields
function GetFields {
[CmdletBinding()]
param($items, $options)
Import-Function GetTemplatesFields
Foreach($item in $items) {
$fields += GetTemplatesFields($item)
}
# Remove any duplicates, based on ID
$fields = $fields | Sort-Object -Property 'Name' -Unique
# build the hashtable to show as checklist options
ForEach ($field in $fields) {
$options.add($field.Name, $field.ID.ToString())
}
return $fields
}
function Translate {
[CmdletBinding()]
param($items, [PSCustomObject]$userOptions)
Write-Host "Proceeding with execution..."
# Call the translator service
$translatorService = New-Object Sitecore.Cognitive.Translator.PSE.Extensions.TranslationExtensions
$items | ForEach-Object {
$currentItem = $_
foreach($lang in $userOptions.ToLanguages) {
Add-ItemLanguage $_ -Language $userOptions.FromLanguage -TargetLanguage $lang -IfExist $userOptions.IfExists
Write-Host "Item : '$($currentItem.Name)' created in language '$lang'"
Get-ItemField -Item $_ -Language $lang -ReturnType Field -Name "*" | ForEach-Object{
# Only look within Single-line and Rich Text fields that has been choosen in the dialog box
if(($_.Type -eq "Single-Line Text" -or $_.Type -eq "Rich Text" -or $_.Type -eq "Multiline Text") -and $userOptions.FieldsToTranslate.Contains($_.ID.ToString())) {
if (-not ([string]::IsNullOrEmpty($_))) {
# Get the item in the target created language
$langItem = Get-Item -Path "master:" -ID $currentItem.ID -Language $lang
# Get the translated content from the service
$translated = $translatorService.TranslateText($currentItem[$_.Name], $userOptions.FromLanguage, $lang, $_.Type)
# edit the item with the translated content
$langItem.Editing.BeginEdit()
$langItem[$_.Name] = $translated.Result
$langItem.Editing.EndEdit()
Write-Host "Field : '$_' translated from '$($userOptions.FromLanguage)'" $currentItem[$_.Name] " to : '$lang'" $translated.Result
}
}
}
}
}
}
In the Translate function, I’m doing the call to the API (Sitecore.Cognitive.Translator.PSE.Extensions.TranslationExtensions).
That’s very much it, now is time to test it! If everything went well, you will be able to add language versions to your items with also translated content from Azure Cognitive Translation.
Let’s see this in action!
For the purpose of this demo, I’ve created a simple content tree with 3 levels, the items has some content in english (plain and HTML) and I’ll be using the tool to create the Spanish-Argentina and French-France versions + translated content.
1- Click on the Home item and choose the Add Language Version and Translate option from the scripts section.
2- Choose the options, in this case I want to translate from the default ‘en‘ language to both ‘es-AR‘ and ‘fr-FR‘. Also I want to include the subitems, but as for this test the items doesn’t have a presentation nor datasources, I’m keeping this disabled. No versions in the target language exist for those items, so I’m keeping the “Skip” option.
3- Click on proceed and choose the fields you want to translate:
I’m selecting all fields, as you can check in the SPE code, I’m removing the standard fields from the items to be translated, normally you don’t want that and it will overpopulate the fields list.
4- Click OK, double check the data entered and click the OK button for making the magic to happen:
5- Click on the View script results link to check the output logs:
6- Check that the items have been created in the desired languages and the contents are already translated. Review them, publish and have a cup of coffee :).
fr-FR items version:
es-AR items version:
Voila! After few clicks you have your content items created in the language version with the content translated, I hope you like it us much as I do.
Find the source code in GitHub, download the Sitecore package here or get the asset image from Docker Hub.
In this post I’m explaining how to switch the blob storage provider to make use of Azure Blob Storage. Before Sitecore 9.3, we could store the blobs on the DB or filesystem, Azure Blob Storage was not supported out of the box and even tough it was possible, it required some customizations to make it working, nowadays, since Sitecore 9.3 a module has been released and is very straightforward to setup, as you will see in this post.
By doing this we can significantly reduce costs and improve performance as the DB size won’t increase that much due to the media library items.
Introduction to Azure Blob storage
Azure Blob storage is Microsoft’s object storage solution for the cloud. Blob storage is optimized for storing massive amounts of unstructured data. Unstructured data is data that doesn’t adhere to a particular data model or definition, such as text or binary data.
Blob storage is designed for:
Serving images or documents directly to a browser.
Storing files for distributed access.
Streaming video and audio.
Writing to log files.
Storing data for backup and restore, disaster recovery, and archiving.
Storing data for analysis by an on-premises or Azure-hosted service.
Users or client applications can access objects in Blob storage via HTTP/HTTPS, from anywhere in the world. Objects in Blob storage are accessible via the Azure Storage REST API, Azure PowerShell, Azure CLI, or an Azure Storage client library.
For more info please refer here and also you can find some good documentation here.
Creating your blob storage resource
Azure Storage Account
Create the resource by following the wizard and then check the “Access Keys” section, you’ll need the “Connection string” later.
Connection String and keys
Configuring your Sitecore instance
There are basically three main option to install the blob storage module into your instance:
Install the Azure Blob Storage module in Sitecore PaaS.
Use the Sitecore Azure Toolkit:
Use a new Sitecore installation with Sitecore Azure Toolkit
Use an existing Sitecore installation with Sitecore Azure Toolkit
Use Sitecore in the Azure Marketplace (for new Sitecore installations only)
Install the Azure Blob Storage module on an on-premise Sitecore instance.
Manually install the Azure Blob Storage module in PaaS or on-premise.
This time I’ll be focusing in the last option, manually installing the module, doesn’t matter if it’s a PaaS or on-premise approach.
7. In the \App_Config\Modules\Sitecore.AzureBlobStorage\Sitecore.AzureBlobStorage.config file, ensure that <param name="blobcontainer"> is the name you gave to the container after creating the resource.
Let’s test it!
If everything went well, then we can just test it by uploading a media item to the Sitecore media library
Let’s have a look now at the Storage Explorer in the Azure portal
Here we go, the image is now uploaded into the Azure blob storage, meaning the config is fine and working as expected.
In my previous post I’ve shared the custom image field implementation that makes use of the Azure Computer Vision service in order to crop and generate the thumbnails using AI. Please before proceed with this reading, make sure you already went through the previous posts: Part I and Part II.
Now, I’ll be sharing the last, but not least part of this topic, how to make it working in the front-end side, the media request flow and so on.
Image request flow
The image request flow
So, the request flow is described in the following graph, basically follows the normal Sitecore flow but with the introduction of the Azure Computer Vision and Image Sharp to generate the proper cropping version of the image.
AICroppingProcessor
This custom processor will be overriding the Sitecore OOTB ThumbnailProcessor. It’s basically a copy from the original code with a customization to check the “SmartCropping” parameter from the image request.
using Sitecore.Diagnostics;
using Sitecore.Resources.Media;
using System;
using Microsoft.Extensions.DependencyInjection;
using System.IO;
using System.Linq;
using Sitecore.Computer.Vision.CroppingImageField.Services;
using Sitecore.DependencyInjection;
namespace Sitecore.Computer.Vision.CroppingImageField.Processors
{
public class AICroppingProcessor
{
private static readonly string[] AllowedExtensions = { "bmp", "jpeg", "jpg", "png", "gif" };
private readonly ICroppingService _croppingService;
public AICroppingProcessor(ICroppingService croppingService)
{
_croppingService = croppingService;
}
public AICroppingProcessor()
{
_croppingService = ServiceLocator.ServiceProvider.GetService<ICroppingService>();
}
public void Process(GetMediaStreamPipelineArgs args)
{
Assert.ArgumentNotNull(args, "args");
var outputStream = args.OutputStream;
if (outputStream == null)
{
return;
}
if (!AllowedExtensions.Any(i => i.Equals(args.MediaData.Extension, StringComparison.InvariantCultureIgnoreCase)))
{
return;
}
var smartCrop = args.Options.CustomOptions[Constants.QueryStringKeys.SmartCropping];
if (!string.IsNullOrEmpty(smartCrop) && bool.Parse(smartCrop))
{
Stream outputStrm;
outputStrm = Stream.Synchronized(_croppingService.GetCroppedImage(args.Options.Width, args.Options.Height, outputStream.MediaItem));
args.OutputStream = new MediaStream(outputStrm, args.MediaData.Extension, outputStream.MediaItem);
}
else if (args.Options.Thumbnail)
{
var transformationOptions = args.Options.GetTransformationOptions();
var thumbnailStream = args.MediaData.GetThumbnailStream(transformationOptions);
if (thumbnailStream != null)
{
args.OutputStream = thumbnailStream;
}
}
}
}
}
We need also to customize the MediaRequest to also take the “SmartCropping” parameter into account:
using Sitecore.Configuration;
using Sitecore.Diagnostics;
using Sitecore.Resources.Media;
using System.Web;
namespace Sitecore.Computer.Vision.CroppingImageField.Requests
{
using System.Collections.Specialized;
public class AICroppingMediaRequest : MediaRequest
{
private HttpRequest _innerRequest;
private MediaUrlOptions _mediaQueryString;
private MediaUri _mediaUri;
private MediaOptions _options;
protected override MediaOptions GetOptions()
{
var queryString = this.InnerRequest.QueryString;
if (queryString == null || queryString.Count == 0)
{
_options = new MediaOptions();
}
else
{
SetMediaOptionsFromMediaQueryString(queryString);
if (!string.IsNullOrEmpty(queryString.Get(Constants.QueryStringKeys.SmartCropping)))
{
SetCustomOptionsFromQueryString(queryString);
}
}
if (!this.IsRawUrlSafe)
{
if (Settings.Media.RequestProtection.LoggingEnabled)
{
string urlReferrer = this.GetUrlReferrer();
Log.SingleError(string.Format("MediaRequestProtection: An invalid/missing hash value was encountered. " +
"The expected hash value: {0}. Media URL: {1}, Referring URL: {2}",
HashingUtils.GetAssetUrlHash(this.InnerRequest.RawUrl), this.InnerRequest.RawUrl,
string.IsNullOrEmpty(urlReferrer) ? "(empty)" : urlReferrer), this);
}
_options = new MediaOptions();
}
return _options;
}
private void SetCustomOptionsFromQueryString(NameValueCollection queryString)
{
this.ProcessCustomParameters(_options);
if (!string.IsNullOrEmpty(queryString.Get(Constants.QueryStringKeys.SmartCropping))
&& !_options.CustomOptions.ContainsKey(Constants.QueryStringKeys.SmartCropping)
&& !string.IsNullOrEmpty(queryString.Get(Constants.QueryStringKeys.SmartCropping)))
{
_options.CustomOptions.Add(Constants.QueryStringKeys.SmartCropping, queryString.Get(Constants.QueryStringKeys.SmartCropping));
}
}
private void SetMediaOptionsFromMediaQueryString(NameValueCollection queryString)
{
MediaUrlOptions mediaQueryString = this.GetMediaQueryString();
_options = new MediaOptions()
{
AllowStretch = mediaQueryString.AllowStretch,
BackgroundColor = mediaQueryString.BackgroundColor,
IgnoreAspectRatio = mediaQueryString.IgnoreAspectRatio,
Scale = mediaQueryString.Scale,
Width = mediaQueryString.Width,
Height = mediaQueryString.Height,
MaxWidth = mediaQueryString.MaxWidth,
MaxHeight = mediaQueryString.MaxHeight,
Thumbnail = mediaQueryString.Thumbnail,
UseDefaultIcon = mediaQueryString.UseDefaultIcon
};
if (mediaQueryString.DisableMediaCache)
{
_options.UseMediaCache = false;
}
foreach (string allKey in queryString.AllKeys)
{
if (allKey != null && queryString[allKey] != null)
{
_options.CustomOptions[allKey] = queryString[allKey];
}
}
}
public override MediaRequest Clone()
{
Assert.IsTrue((base.GetType() == typeof(AICroppingMediaRequest)), "The Clone() method must be overridden to support prototyping.");
return new AICroppingMediaRequest
{
_innerRequest = this._innerRequest,
_mediaUri = this._mediaUri,
_options = this._options,
_mediaQueryString = this._mediaQueryString
};
}
}
}
This code is very straightforward, it will basically check if the “SmartCropping=true” parameter exists in the media request, and then executes the custom code to crop the image.
The “Get Thumbnails” method limitations
As we can see in the official documentation, there are some limitations on the thumbnail generator method.
Image file size must be less than 4MB.
Image dimensions should be greater than 50 x 50.
Width of the thumbnail must be between 1 and 1024.
Height of the thumbnail must be between 1 and 1024.
The most important one is that the width and height cannot exceed the 1024px, this is problematic as sometimes we need to crop on a bigger ratio.
So, in order to make it more flexible, I’m doing the cropping using the Graphics library but getting the focus point coordinates from the “Get Area Of Interest” API method:
using Sitecore.Data.Items;
using Microsoft.Extensions.DependencyInjection;
using System.IO;
using Sitecore.DependencyInjection;
using Sitecore.Resources.Media;
using System.Drawing;
using System.Drawing.Imaging;
using System.Drawing.Drawing2D;
namespace Sitecore.Computer.Vision.CroppingImageField.Services
{
public class CroppingService : ICroppingService
{
private readonly ICognitiveServices _cognitiveServices;
public CroppingService(ICognitiveServices cognitiveServices)
{
_cognitiveServices = cognitiveServices;
}
public CroppingService()
{
_cognitiveServices = ServiceLocator.ServiceProvider.GetService<ICognitiveServices>();
}
public Stream GetCroppedImage(int width, int height, MediaItem mediaItem)
{
using (var streamReader = new MemoryStream())
{
var mediaStrm = mediaItem.GetMediaStream();
mediaStrm.CopyTo(streamReader);
mediaStrm.Position = 0;
var img = Image.FromStream(mediaStrm);
// The cropping size shouldn't be higher than the original image
if (width > img.Width || height > img.Height)
{
Sitecore.Diagnostics.Log.Warn($"Media file is smaller than the requested crop size. " +
$"This can result on a low quality result. Please upload a proper image: " +
$"Min Height:{height}, Min Width:{width}. File: {mediaItem.DisplayName}, Path{mediaItem.MediaPath}", this);
}
// if the cropping size exceeds the cognitive services limits, get the focus point and crop
if (width > 1025 || height > 1024)
{
var area = _cognitiveServices.GetAreaOfImportance(streamReader.ToArray());
var cropImage = CropImage(img, area.areaOfInterest.X, area.areaOfInterest.Y, width, height);
return cropImage;
}
var thumbnailResult = _cognitiveServices.GetThumbnail(streamReader.ToArray(), width, height);
return new MemoryStream(thumbnailResult);
}
}
public string GenerateThumbnailUrl(int width, int height, MediaItem mediaItem)
{
var streamReader = MediaManager.GetMedia(mediaItem).GetStream();
{
using (var memStream = new MemoryStream())
{
streamReader.Stream.CopyTo(memStream);
var thumbnail = _cognitiveServices.GetThumbnail(memStream.ToArray(), width, height);
var imreBase64Data = System.Convert.ToBase64String(thumbnail);
return $"data:image/png;base64,{imreBase64Data}";
}
}
}
private Stream CropImage(Image source, int x, int y, int width, int height)
{
var bmp = new Bitmap(width, height);
var outputStrm = new MemoryStream();
using (var gr = Graphics.FromImage(bmp))
{
gr.InterpolationMode = InterpolationMode.HighQualityBicubic;
using (var wrapMode = new ImageAttributes())
{
wrapMode.SetWrapMode(WrapMode.TileFlipXY);
gr.DrawImage(source, new Rectangle(0, 0, bmp.Width, bmp.Height), x, y, width, height, GraphicsUnit.Pixel, wrapMode);
}
}
bmp.Save(outputStrm, source.RawFormat);
return outputStrm;
}
}
}
Let’s see this in action!
After picking your picture in the AI Cropping Image field, it gets already cropped and you can see the different thumbnails. You can choose or change the thumbnails by updating the child items here: /sitecore/system/Settings/Foundation/Vision/Thumbnails.
Also note that you get an auto generated Alt text “Diego Maradona holding a ball” and a list of tags.
AI Cropping Image Field
The results
This is how the different cropped images will look like in the front end. Depending on your front end implementation, you will define different cropping sizes per breakpoints.
In this following implementation, I’m setting the image as a background and using the option to render the image URL as follows:
<img alt="a close up of a person wearing glasses"
src="https://vision.test.cm/-/media/project/vision/homepage/iatestimage.png?
w=600&h=600&smartCropping=true&hash=C2E215FE2CF74D4C8142E35619ABB8DE">
Note: Have a look at the AdvancedImageParameters:
OnlyUrl: If true it will just render the image URL (for being used as src in the img tag).
AutoAltText: If true, the alt text will be replaced by the one generated from Azure IA.
Width and Height: int values, to specify the cropping size.
Widths and Sizes: If set, it will generate a srcset image with for the different breakpoints.
SizesTag and SrcSetTag: Those are mandatories if when using the previous settings.
<img alt="a close up of a person wearing glasses" data-sizes="50vw,(min-width:
999px) 25vw,(min-width: 1200px) 15vw" data-
srcset="https://vision.test.cm/-/media/project/vision/homepage/iatestimage.png?
w=170&hash=1D04C1F551E9606AB2EEB3C712255651
170w,https://vision.test.cm/-/media/project/vision/homepage/iatestimage.png?
w=233&hash=DD2844D340246D3CF8AEBB63CE4E9397
233w,https://vision.test.cm/-/media/project/vision/homepage/iatestimage.png?
w=340&hash=3B773ACB5136214979A0009E24F25F02
340w,https://vision.test.cm/-/media/project/vision/homepage/iatestimage.png?
w=466&hash=424F7615FBECFED21F48DA0AE1FE7A5B 466w"
src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==">
GlassMapper extension
At last, an extension method has been added in order to get the media URL from the image field.
In my previous post I’ve shared a quick overview on the Azure Computer Vision API service and it’s implementation. If you didn’t read it yet, please do before proceeding to this reading!
With the basics and the CognitiveServices in place, let’s move forward and create a custom image field that uses this service to handle the image cropping, tagging and alt text description, all with AI.
I’ll be sharing the whole implementation in GitHub later and also a package plugin, but let’s get into the implementation details first.
Custom Image Field
The first step is to create the custom field, for doing that, go to the core DB and duplicate the /sitecore/system/Field types/Simple Types/Image field item. Let’s call it “AICroppedImage“.
Keep everything as it is except the assembly and class fields
AICroppedImage Class
For the implementation, we just decompiled the code from Sitecore.Kernel (Sitecore.Shell.Applications.ContentEditor.Image) and made all our needed customizations.
using Sitecore.Configuration;
using Sitecore.Data.Items;
using Sitecore.DependencyInjection;
using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.Resources.Media;
using Sitecore.Shell.Applications.ContentEditor;
using Sitecore.Web.UI.Sheer;
using System;
using System.IO;
using System.Text;
using System.Web;
using System.Web.UI;
using Microsoft.Extensions.DependencyInjection;
using System.Linq;
using Sitecore.Computer.Vision.CroppingImageField.Models.ImagesDetails;
using Sitecore.Computer.Vision.CroppingImageField.Services;
namespace Sitecore.Computer.Vision.CroppingImageField.Fields
{
public class AICroppedImage : Image
{
private readonly string ThumbnailsId = Settings.GetSetting("Sitecore.Computer.Vision.CroppingImageField.AICroppingField.ThumbnailsFolderId");
private readonly ICognitiveServices _cognitiveServices;
private readonly ICroppingService _croppingService;
public AICroppedImage(ICognitiveServices cognitiveServices, ICroppingService croppingService) : base()
{
_cognitiveServices = cognitiveServices;
_croppingService = croppingService;
}
public AICroppedImage() : base()
{
_cognitiveServices = ServiceLocator.ServiceProvider.GetService<ICognitiveServices>();
_croppingService = ServiceLocator.ServiceProvider.GetService<ICroppingService>();
}
protected override void DoRender(HtmlTextWriter output)
{
Assert.ArgumentNotNull((object)output, nameof(output));
Item mediaItem = this.GetMediaItem();
string src;
this.GetSrc(out src);
string str1 = " src=\"" + src + "\"";
string str2 = " id=\"" + this.ID + "_image\"";
string str3 = " alt=\"" + (mediaItem != null ? HttpUtility.HtmlEncode(mediaItem["Alt"]) : string.Empty) + "\"";
this.Attributes["placeholder"] = Translate.Text(this.Placeholder);
string str = this.Password ? " type=\"password\"" : (this.Hidden ? " type=\"hidden\"" : "");
this.SetWidthAndHeightStyle();
output.Write("<input" + this.ControlAttributes + str + ">");
this.RenderChildren(output);
output.Write("<div id=\"" + this.ID + "_pane\" class=\"scContentControlImagePane\">");
string clientEvent = Sitecore.Context.ClientPage.GetClientEvent(this.ID + ".Browse");
output.Write("<div class=\"scContentControlImageImage\" onclick=\"" + clientEvent + "\">");
output.Write("<iframe" + str2 + str1 + str3 + " frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" width=\"100%\" height=\"128\" " +
"allowtransparency=\"allowtransparency\"></iframe>");
output.Write("<div id=\"" + this.ID + "_thumbnails\">");
output.Write(GetThumbnails());
output.Write("</div>");
output.Write("</div>");
output.Write("<div>");
output.Write("<div id=\"" + this.ID + "_details\" class=\"scContentControlImageDetails\">");
string details = this.GetDetails();
output.Write(details);
output.Write("</div>");
output.Write("</div>");
}
protected override void DoChange(Message message)
{
Assert.ArgumentNotNull((object)message, nameof(message));
base.DoChange(message);
if (Sitecore.Context.ClientPage.Modified)
{
this.Update();
}
if (string.IsNullOrEmpty(this.Value))
{
this.ClearImage();
}
SheerResponse.SetReturnValue(true);
}
protected new void BrowseImage(ClientPipelineArgs args)
{
Assert.ArgumentNotNull((object)args, nameof(args));
base.BrowseImage(args);
if (Sitecore.Context.ClientPage.Modified)
{
this.Update();
}
}
protected new void ShowProperties(ClientPipelineArgs args)
{
Assert.ArgumentNotNull((object)args, nameof(args));
base.ShowProperties(args);
if (Sitecore.Context.ClientPage.Modified)
{
this.Update();
}
}
public override void HandleMessage(Message message)
{
Assert.ArgumentNotNull((object)message, nameof(message));
base.HandleMessage(message);
string name = message.Name;
if (name == "contentimage:clear")
{
this.ClearImage();
}
else if (name == "contentimage:refresh")
{
this.Update();
}
}
private void ClearImage()
{
if (this.Disabled)
{
return;
}
if (this.Value.Length > 0)
{
this.SetModified();
}
this.XmlValue = new XmlValue(string.Empty, "image");
this.Value = string.Empty;
this.Update();
}
protected new void Update()
{
string src;
this.GetSrc(out src);
SheerResponse.SetAttribute(this.ID + "_image", "src", src);
SheerResponse.SetInnerHtml(this.ID + "_thumbnails", this.GetThumbnails());
SheerResponse.SetInnerHtml(this.ID + "_details", this.GetDetails());
SheerResponse.Eval("scContent.startValidators()");
}
private string GetDetails()
{
var empty = string.Empty;
MediaItem mediaItem = this.GetMediaItem();
if (mediaItem != null)
{
var innerItem = mediaItem.InnerItem;
var stringBuilder = new StringBuilder();
var xmlValue = this.XmlValue;
stringBuilder.Append("<div>");
var item = innerItem["Dimensions"];
var str = HttpUtility.HtmlEncode(xmlValue.GetAttribute("width"));
var str1 = HttpUtility.HtmlEncode(xmlValue.GetAttribute("height"));
ImageDetails imageDetails;
using (var streamReader = new MemoryStream())
{
var mediaStrm = mediaItem.GetMediaStream();
mediaStrm.CopyTo(streamReader);
imageDetails = _cognitiveServices.AnalyzeImage(streamReader.ToArray());
}
if (!string.IsNullOrEmpty(str) || !string.IsNullOrEmpty(str1))
{
var objArray = new object[] { str, str1, item };
stringBuilder.Append(Translate.Text("Dimensions: {0} x {1} (Original: {2})", objArray));
}
else
{
var objArray1 = new object[] { item };
stringBuilder.Append(Translate.Text("Dimensions: {0}", objArray1));
}
stringBuilder.Append("</div>");
stringBuilder.Append("<div style=\"padding:2px 0px 0px 0px; text-align=left; \">");
var str2 = HttpUtility.HtmlEncode(innerItem["Alt"]);
var str3 = imageDetails.Description.Captions.FirstOrDefault()?.Text;
if (!string.IsNullOrEmpty(str3) && !string.IsNullOrEmpty(str2))
{
var objArray2 = new object[] { str3, str2 };
stringBuilder.Append(Translate.Text("AI Alternate Text: \"{0}\" (Default Alternate Text: \"{1}\")", objArray2));
}
else if (!string.IsNullOrEmpty(str3))
{
var objArray3 = new object[] { str3 };
stringBuilder.Append(Translate.Text("AI Alternate Text: \"{0}\"", objArray3));
}
else
{
var objArray4 = new object[] { str2 };
stringBuilder.Append(Translate.Text("Default Alternate Text: \"{0}\"", objArray4));
}
stringBuilder.Append("</br>");
var objArray5 = new object[] { str3 };
stringBuilder.Append(Translate.Text("Tags: \"{0}\"", string.Join(",", imageDetails.Description.Tags), objArray5));
stringBuilder.Append("</div>");
empty = stringBuilder.ToString();
}
if (empty.Length == 0)
{
empty = Translate.Text("This media item has no details.");
}
return empty;
}
private Item GetMediaItem()
{
var attribute = this.XmlValue.GetAttribute("mediaid");
if (attribute.Length <= 0)
{
return null;
}
Language language = Language.Parse(this.ItemLanguage);
return Sitecore.Client.ContentDatabase.GetItem(attribute, language);
}
private MediaItem GetSrc(out string src)
{
src = string.Empty;
MediaItem mediaItem = (MediaItem)this.GetMediaItem();
if (mediaItem == null)
{
return null;
}
var thumbnailOptions = MediaUrlOptions.GetThumbnailOptions(mediaItem);
int result;
if (!int.TryParse(mediaItem.InnerItem["Height"], out result))
{
result = 128;
}
thumbnailOptions.Height = Math.Min(128, result);
thumbnailOptions.MaxWidth = 640;
thumbnailOptions.UseDefaultIcon = true;
src = MediaManager.GetMediaUrl(mediaItem, thumbnailOptions);
return mediaItem;
}
private string GetThumbnails()
{
var html = new StringBuilder();
var src = string.Empty;
var mediaItem = this.GetSrc(out src);
if (mediaItem == null)
{
return string.Empty;
}
html.Append("<ul id=" + this.ID + "_frame\" style=\"display: -ms-flexbox;display: flex;-ms-flex-direction: row;flex-direction: row;-ms-flex-wrap: wrap;flex-wrap: wrap;\">");
var thumbnailFolderItem = Sitecore.Client.ContentDatabase.GetItem(new Sitecore.Data.ID(ThumbnailsId));
if (thumbnailFolderItem != null && thumbnailFolderItem.HasChildren)
{
foreach (Item item in thumbnailFolderItem.Children)
{
GetThumbnailHtml(item, html, mediaItem);
}
}
html.Append("</ul>");
return html.ToString();
}
private void GetThumbnailHtml(Item item, StringBuilder html, MediaItem mediaItem)
{
if (item.Fields["Size"]?.Value != null)
{
var values = item.Fields["Size"].Value.Split('x');
var width = values[0];
var height = values[1];
int w, h;
if (int.TryParse(width, out w) && Int32.TryParse(height, out h) && w > 0 && h > 0)
{
var imageSrc = _croppingService.GenerateThumbnailUrl(w, h, mediaItem);
html.Append(string.Format("<li id=\"Frame_{0}_{1}\" style=\"width: {2}px; height: {3}px; position: relative; overflow: hidden; display: inline-block;border: solid 3px #fff;margin: 5px 5px 5px 0;\">" +
"<img style=\"position: relative;position: absolute;left: 0;top: 0;margin: 0;display: block;width: auto; height: auto;min-width: 100%; min-height: 100%;max-height: none; max-width: none;\" " +
"src=\"{4}\"><img /><span style=\"position: absolute;" +
"top: 0;left: 0;padding: 2px 3px;background-color: #fff;opacity: 0.8;\">{5}</span></li>", this.ID, item.ID.ToShortID(), w, h, imageSrc, item.DisplayName));
}
}
}
}
}
We’re basically modifying the way Sitecore renders the field with some small customizations, basically to add the thumbnails generated by the Azure Cognitive service and also the Alt and Tags texts.
Ok, so that’s very much it, let’s deploy our code and see how it looks in the Sitecore Content Editor. The only thing you need to do next, is create a template and make use of the newly created “AI Cropped Image” field.
Et Voila! The image field is now rendering a few thumbnails that gives you an idea of the final results when rendering the image in the front-end. As you can see, it gives also some tags and a description (“Diego Maradona holding a ball”) used as alt text, everything coming from the Azure AI service, awesome!
Make the field rendered to work as an OOTB Sitecore image field
Next step, is to make sure we can still using the Sitecore helpers for rendering this field. For making this possible, we want to customize the Sitecore.Pipelines.RenderField.GetImageFieldValue processor. Same as before, we decompile the OOTB code from Sitecore.Kernel and we make our updates there. Then just patch the config like that:
Here, we just need to add the newly created field type (AI Cropped Image) as a valid image field type by overriding the IsImage() method.
using Sitecore.Diagnostics;
using Sitecore.Pipelines.RenderField;
namespace Sitecore.Computer.Vision.CroppingImageField.Pipelines
{
public class RenderAICroppingImageField : GetImageFieldValue
{
public override void Process(RenderFieldArgs args)
{
Assert.ArgumentNotNull((object)args, nameof(args));
if (!this.IsImage(args))
{
return;
}
var renderer = this.CreateRenderer();
this.ConfigureRenderer(args, renderer);
this.SetRenderFieldResult(renderer.Render(), args);
}
protected override bool IsImage(RenderFieldArgs args)
{
return args.FieldTypeKey == "AI Cropped Image";
}
}
}
Make it working with GlassMapper
Now, we can do some quick updates to GlassMapper as well so we can benefit from the glass helpers. Let’s add a custom field mapper, again after decompiling Glass.Mapper.Sc.DataMappers.SitecoreFieldImageMapper, we can just extend it to work in the same way with the newly introduced AI Cropping Image field.
using Glass.Mapper.Sc;
using Glass.Mapper.Sc.Configuration;
using Glass.Mapper.Sc.DataMappers;
using Sitecore.Data;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using System;
using Sitecore.Computer.Vision.CroppingImageField.Fields;
namespace Sitecore.Computer.Vision.CroppingImageField.Mappers
{
public class AICroppedImageFieldMapper : AbstractSitecoreFieldMapper
{
public AICroppedImageFieldMapper(): base(typeof(AICroppedImage))
{
}
public override object GetField(Field field, SitecoreFieldConfiguration config, SitecoreDataMappingContext context)
{
var img = new AICroppedImage();
var sitecoreImage = new AICroppedImageField(field);
SitecoreFieldImageMapper.MapToImage(img, sitecoreImage);
return img;
}
public override void SetField(Field field, object value, SitecoreFieldConfiguration config, SitecoreDataMappingContext context)
{
var img = value as AICroppedImage;
if (field == null || img == null)
{
return;
}
var item = field.Item;
var sitecoreImage = new AICroppedImageField(field);
SitecoreFieldImageMapper.MapToField(sitecoreImage, img, item);
}
public override string SetFieldValue(object value, SitecoreFieldConfiguration config, SitecoreDataMappingContext context)
{
throw new NotImplementedException();
}
public override object GetFieldValue(string fieldValue, SitecoreFieldConfiguration config, SitecoreDataMappingContext context)
{
var item = context.Service.Database.GetItem(new ID(fieldValue));
if (item == null)
{
return null;
}
var imageItem = new MediaItem(item);
var image = new AICroppedImage();
SitecoreFieldImageMapper.MapToImage(image, imageItem);
return image;
}
}
}
We need also to create our custom field that inherits from Glass.Mapper.Sc.Fields.Image
using Glass.Mapper.Sc.Fields;
namespace Sitecore.Computer.Vision.CroppingImageField.Mappers
{
public class AICroppedImage : Image
{
}
}
Last step is to add the mapper to the create resolver from the GlassMapperSCCustom.cs
public static class GlassMapperScCustom
{
public static IDependencyResolver CreateResolver(){
var config = new Glass.Mapper.Sc.Config();
var dependencyResolver = new DependencyResolver(config);
// add any changes to the standard resolver here
dependencyResolver.DataMapperFactory.First(() => new AICroppedImageFieldMapper());
dependencyResolver.Finalise();
return dependencyResolver;
}
}
Custom Caching
In order to reduce the calls to the service, an extra layer of caching has been implemented. This cache, as any other Sitecore cache gets flushed after a publishing and the size can be easily configured through it’s configuration.
In my next post, I’ll be sharing the front end implementation, the full media request flow and the customizations needed to make it working in your site. Stay tuned!
Images are nowadays a critical part on websites, specially with “mobile first” approach and responsive designs. Cropping images in a proper way is extremely important if you don’t want to destroy your website’s user experience. Imagine an e-commerce website offering a product that is not visible anymore when the user is browsing the site from a mobile device.
In this post I’ll share a way to solve this issue with the help of AI, more specifically using Azure Cognitive Services (Computer Vision).
Azure Computer Vision
The Computer Vision API provides state-of-the-art algorithms to process images and return information. For example, it can be used to determine if an image contains mature content, or it can be used to find all the faces in an image. It also has other features like estimating dominant and accent colors, categorizing the content of images, and describing an image with complete English sentences. Additionally, it can also intelligently generate images thumbnails for displaying large images effectively. For more details about the API, refer to the official documentation here. It gives also some some good examples in C#.
First Step: Create the Azure resource
Before being able to play with this awesome service, we’ve to create the resource, and good news: the free plan would be enough for your tests (20 calls/min – 5K calls/month):
Login to the Azure portal, and go to add a new resource, search for “Computer Vision” and as usual, follow the wizard in order to create it.
Then just go to the “Keys and Endpoint” section and get your key, endpoint and location. Let’s write those down, we’ll use later to connect to our API.
For this implementation I’ll be using the following methods:
Analyze Image: This operation extracts a rich set of visual features based on the image content.
Get Area of Interest: This operation returns a bounding box around the most important area of the image.
Get Thumbnail: This operation generates a thumbnail image with the user-specified width and height. By default, the service analyzes the image, identifies the region of interest (ROI), and generates smart cropping coordinates based on the ROI. Smart cropping helps when you specify an aspect ratio that differs from that of the input image.
Testing the endopins
We can now use Postman for testing the API endpoints and the results we get. This is very straightforward by following the documentation from MS:
Do a POST or GET (depending on the service you want tot test), to the following URL: https://{yourComputerVisionService}.cognitiveservices.azure.com/vision/v2.0/{APIMethod}?{Params}
Add the needed headers:
Ocp-Apim-Subscription-Key: Your app key from the “Keys and Endpoint” previous section.
Ocp-Apim-Subscription-Region: Your app region from the “Keys and Endpoint” previous section.
Content-Type: application/json
Add the URL of the image in the “Body“.
Let’s do a test with the following image:
Get Thumbnail
As you can see, Computer Vision is retrieving a cropped version of the image by the width/height we passed as parameters (200×200). And it’s cropping in the right way keeping focus in the most important part of the picture.
Get Area Of Interest
Same as generating the thumbnail, it retrieves the coordinates of the area of interest. As the thumbnail generation has some limitations that I’ll explain later, I’ll be using this method to crop the image.
Analyze Image
Depending on the parameters we send to this method, it will return a lot of different elements after analyzing the image, such as tags, description, brands information, etc. I’ll be using this method for generating tags but also to give an automatic alt text to the image.
Service Implementation
Let’s now implement the API service. As an starting point, we’ve to create a service that will take care of the communication to the Computer Vision API:
The ICognitiveServices Interface:
using Sitecore.Computer.Vision.CroppingImageField.Models.AreaOfInterest;
using Sitecore.Computer.Vision.CroppingImageField.Models.ImagesDetails;
namespace Sitecore.Computer.Vision.CroppingImageField.Services
{
public interface ICognitiveServices
{
ImageDetails AnalyzeImage(byte[] image);
byte[] GetThumbnail(byte[] image, int width, int height);
AreaOfInterestResult GetAreaOfImportance(byte[] image);
}
}
The CognitiveServices Class:
using System;
using Newtonsoft.Json;
using Sitecore.Configuration;
using Sitecore.Diagnostics;
using System.Net.Http;
using System.Net.Http.Headers;
using Sitecore.Computer.Vision.CroppingImageField.Models.AreaOfInterest;
using Sitecore.Computer.Vision.CroppingImageField.Models.ImagesDetails;
using Sitecore.Computer.Vision.CroppingImageField.Caching;
using Sitecore.Computer.Vision.CroppingImageField.Extensions;
namespace Sitecore.Computer.Vision.CroppingImageField.Services
{
public class CognitiveServices : ICognitiveServices
{
private readonly string _cognitiveServicesKey = Settings.GetSetting($"Sitecore.Computer.Vision.CroppingImageField.AICroppingField.CognitiveServices.ApiKey", "");
private readonly string _cognitiveServicesUrl = Settings.GetSetting($"Sitecore.Computer.Vision.CroppingImageField.AICroppingField.CognitiveServices.ApiUrl", "");
private readonly string _cognitiveServicesZone = Settings.GetSetting($"Sitecore.Computer.Vision.CroppingImageField.AICroppingField.CognitiveServices.ApiZone", "");
public ImageDetails AnalyzeImage(byte[] image)
{
var requestUri = _cognitiveServicesUrl + "analyze?" + Settings.GetSetting(
$"Sitecore.Computer.Vision.CroppingImageField.AICroppingField.CognitiveServices.Analyze.Parameters", "");
return CacheManager.GetCachedObject(image.GetHashKey() + requestUri, () =>
{
using (var response = this.CallApi(image, requestUri))
{
if (response.IsSuccessStatusCode)
{
var result = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
var responeData =
JsonConvert.DeserializeObject<ImageDetails>(result, new JsonSerializerSettings());
return responeData;
}
var errorMessage = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
Log.Error(errorMessage, this);
return null;
}
});
}
public byte[] GetThumbnail(byte[] image, int width, int height)
{
var requestUri = _cognitiveServicesUrl +
$"generateThumbnail?width={width}&height={height}&{Constants.QueryStringKeys.SmartCropping}=true";
return CacheManager.GetCachedObject(image.GetHashKey() + requestUri, () =>
{
using (var response = this.CallApi(image, requestUri))
{
if (response.IsSuccessStatusCode)
{
return response.Content.ReadAsByteArrayAsync().GetAwaiter().GetResult();
}
var errorMessage = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
Log.Error(errorMessage, this);
return null;
}
});
}
public AreaOfInterestResult GetAreaOfImportance(byte[] image)
{
var requestUri = _cognitiveServicesUrl + "areaOfInterest";
return CacheManager.GetCachedObject(image.GetHashKey() + requestUri, () =>
{
using (var response = this.CallApi(image, requestUri))
{
if (response.IsSuccessStatusCode)
{
var result = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
var responeData = JsonConvert.DeserializeObject<AreaOfInterestResult>(result, new JsonSerializerSettings());
return responeData;
}
var errorMessage = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
Log.Error(errorMessage, this);
return null;
}
});
}
private HttpResponseMessage CallApi(byte[] image, string requestUri)
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", _cognitiveServicesKey);
client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Region", _cognitiveServicesZone);
using (var content = new ByteArrayContent(image))
{
content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
return client.PostAsync(requestUri, content).GetAwaiter().GetResult();
}
}
}
}
}
So, now we have our Azure Computer Vision resource created, our code is ready and we can connect and play with it.
In the next post, I’ll be creating a custom Sitecore image field that makes use of this implementation to solves the cropping issues and also adds the alt text automatically generated to the image. I’ll be sharing the code in GitHub but also a plugin package, stay tuned!
If you’re working on a Sitecore project that is hosted in Azure PaaS, surely you have found yourself in situations where there is a bug in production or UAT that you cannot easily reproduce in your local environment. Adding logging can also help in such situation, but sometimes this is not enough to troubleshoot your issue. So, debugging your Azure webapp instance seems to be the best and quick option to go and fix your bug!
First step, connect to Cloud Explorer
In Visual Studio, go to “main menu” -> “view” -> “Cloud Explorer“. You’ll get a list of resources that you have access within your Azure subscription. Search for the “App Services” section and choose the webApp you want to attach the debugger to. Note: choose the group by “Resource Types” option to make it easier to navigate.
You’ll get there the list of different resources types, this time expand the “App Services” as this is where your Sitecore webApp should show up.
Before debugging
In order to be able to debug our code, of course we need our dll to be compiled in “debug” mode. So, as the deployed instance must be compiled in “release” mode so you have to first override the dll you want to attach the debugger to.
Make sure you’re doing it with the same code base version that is deployed into the server you’re going to test.
We’ll also need to have the debugging symbols (*.pdb file) in place, so basically if you are debugging a feature called “MyProject.Feature.Navigation“, then make sure you move those files to your webApp (“/bin” folder):
MyProject.Feature.Navigation.dll (debug)
MyProject.Feature.Navigation.pdb
Note: you can use the same Cloud Explorer to copy those files to the server!
Web App settings
In order to make it working, make sure that the remote debugging is enabled on your web app.
Go to the configuration sections and then “General Settings”:
It’s very simple, just enable it and choose the VS version you’ll be using for debugging.
Let’s debug!
So, as you can see the process is easy and very straightforward, there are several other ways for remote debugging a webApp, but the one I’m explaining here is the easiest one.
Your setup is already done, you can now add a breakpoint to the code you want to debug, (make sure you’re in the same code version that you built in debug mode in the previous step, otherwise you’ll get the message saying the breakpoint cannot be attached).
So, that’s it! You should now be able to troubleshoot and fix your issue on an Azure webApp.
In this post I’ll share how to use Azure Redis Cache as Sitecore custom cache provider.
Azure Cache for Redis is a fully managed, distributed, in-memory cache that enables high-performance and scalable architectures. You can use it to create cloud or hybrid deployments that handle millions of requests per second at sub-millisecond latency, all with the configuration, security and availability benefits of a managed service. More info here.
The first step is to create the Redis cache in Azure, for this we log in to the Azure Portal and then add a new resource, search for “Azure Cache for Redis” and choose a plan, for this demo I selected a “Basic C1” plan, we can scale it later if needed.
Azure Redis Cache is now deployed and ready to connect to.
The next step is to get the connection string data and add a new entry “redis.sessions” into the connectionstrings.config file:
Now our app is connected to the Redis cache. Let’s now have a look at a custom cache implementation.
We start by creating a cache provider:
[Service(typeof(IRedisCacheProvider), Lifetime = Lifetime.Singleton)]
public class RedisCacheProvider : IRedisCacheProvider
{
private static readonly Lazy<ConnectionMultiplexer> LazyConnection = new Lazy<ConnectionMultiplexer>(() =>
{
var connectionString = ConfigurationManager.ConnectionStrings["redis.sessions"].ConnectionString;
var options = ConfigurationOptions.Parse(connectionString);
options.AllowAdmin = true;
options.SyncTimeout = 60000;
options.ConnectRetry = 5;
return ConnectionMultiplexer.Connect(options);
});
public static ConnectionMultiplexer Connection => LazyConnection.Value;
private readonly IDatabase _redisCache;
public RedisCacheProvider()
{
_redisCache = Connection.GetDatabase();
}
public IDatabase GetRedisCache()
{
return _redisCache;
}
public IServer GetServer()
{
return Connection.GetServer(Connection.GetEndPoints().FirstOrDefault());
}
}
Now we need to a create a cache manager, that class will contain all the methods to call the cache and to communicate with Redis:
[Service(typeof(ICacheManager), Lifetime = Lifetime.Singleton)]
public class CacheManager : ICacheManager
{
private readonly IDatabase _redisCache;
private readonly IServer _redisServer;
public CacheManager(IRedisCacheProvider redisCacheProvider)
{
_redisCache = redisCacheProvider.GetRedisCache();
_redisServer = redisCacheProvider.GetServer();
}
private static readonly Dictionary<string, object> CacheKeyDictionary = new Dictionary<string, object>();
public object Get(string key)
{
return Get(key, string.Empty);
}
public object Get(string key, string site)
{
var siteName = string.IsNullOrEmpty(site) ? Context.Site?.Name : site;
var cacheKey = $"{siteName}{Context.Database?.Name}{Context.Language}{key}";
var res = _redisCache.StringGet(cacheKey);
return !string.IsNullOrEmpty(res) ? JsonConvert.DeserializeObject(res) : res;
}
public void Set(string key, object value)
{
Set(key, value, string.Empty);
}
public void Set(string key, object value, string site)
{
var siteName = string.IsNullOrEmpty(site) ? Context.Site?.Name : site;
var cacheKey = $"{siteName}{Context.Database?.Name}{Context.Language}{key}";
_redisCache.StringSet(cacheKey, JsonConvert.SerializeObject(value));
}
public IList<string> GetAllKeys()
{
return _redisServer.Keys().Select(k => k.ToString()).ToList();
}
public void Remove(string key)
{
_redisCache.KeyDelete(key);
}
public void ClearCache(object sender, EventArgs args)
{
Log.Info($"RedisCache Cache Clearer.", this);
_redisServer.FlushAllDatabases();
Log.Info("RedisCache Cache Clearer done.", (object)this);
}
public TObj GetCachedObject<TObj>(string cacheKey, Func<TObj> creator) where TObj : class
{
return GetCachedObject(cacheKey, creator, string.Empty);
}
public TObj GetCachedObject<TObj>(string cacheKey, Func<TObj> creator, string site) where TObj : class
{
if (string.IsNullOrEmpty(site))
{
site = Context.Site.Name;
}
var obj = Get(cacheKey, site) as TObj;
if (obj == null)
{
// get the lock object
var lockObject = GetCacheLockObject(cacheKey, site);
try
{
lock (lockObject)
{
obj = creator.Invoke();
Set(cacheKey, obj);
}
}
finally
{
RemoveCacheLockObject(cacheKey, site);
}
}
return obj;
}
private object GetCacheLockObject(string cacheKey, string site)
{
cacheKey += site;
lock (CacheKeyDictionary)
{
if (!CacheKeyDictionary.ContainsKey(cacheKey))
{
CacheKeyDictionary.Add(cacheKey, new object());
}
return CacheKeyDictionary[cacheKey];
}
}
private void RemoveCacheLockObject(string cacheKey, string site)
{
cacheKey += site;
lock (CacheKeyDictionary)
{
if (CacheKeyDictionary.ContainsKey(cacheKey))
{
CacheKeyDictionary.Remove(cacheKey);
}
}
}
}
It’s important to keep in mind that this is a distributed cache, meaning that all Sitecore instances connected to the same cache are sharing it, for example, if we’ve a setup with one CM instance and two CDs, all of those will be sharing the same cache, while in memory cache is specific to the instance. That’s why I’m adding the site name, database and language to the cache key.
Almost done, but now we have to think about one of the most important things when working with caches, when and how to invalidatethose.
We can just call the ClearCache()on the publish:end and publish:end:remote events, but I wanted to make it a bit flexible, as the cache is shared across instances is better to keep control on that rather than just flushing everything on each publish action.
I decided to go with a custom event handler approach. Check the config patch, I’m introducing the customCache:rebuild and customCache:rebuild:remote events:
<!--For more information on using transformations see the web.config examples at http://go.microsoft.com/fwlink/?LinkId=214134. -->
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set" xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<sitecore>
<pipelines>
<initialize>
<processor type="Foundation.RedisCache.Pipelines.Initialize, Foundation.RedisCache" method="InitializeFromPipeline" />
</initialize>
</pipelines>
<commands>
<command name="rediscache:cleancache" type="Foundation.RedisCache.Commands.CleanCacheCommand, Foundation.RedisCache" />
</commands>
<events xdt:Transform="Insert">
<event name="customCache:rebuild">
<handler type="Foundation.RedisCache.Events.EventHandlers.CacheRebuildEventHandler, Foundation.RedisCache" method="OnCustomCacheRebuild" />
</event>
<event name="customCache:rebuild:remote">
<handler type="Foundation.RedisCache.Events.EventHandlers.CacheRebuildEventHandler, Foundation.RedisCache" method="OnCustomCacheRebuild" />
</event>
</events>
</sitecore>
</configuration>
The initialize pipeline:
public class Initialize
{
/// <summary>
/// Initializes event subscription
/// </summary>
/// <param name="args">Args</param>
public virtual void InitializeFromPipeline(PipelineArgs args)
{
var action = new Action<CacheRebuildEvent>(RaiseRemoteEvent);
Sitecore.Eventing.EventManager.Subscribe<CacheRebuildEvent>(action);
}
/// <summary>
/// Raises remote event
/// </summary>
/// <param name="cacheRebuildEvent"></param>
private void RaiseRemoteEvent(CacheRebuildEvent cacheRebuildEvent)
{
var eventArgs = new object[] { new CacheRebuildEventArgs(cacheRebuildEvent) };
Sitecore.Events.Event.RaiseEvent(Constants.CustomCacheRebuildEventNameRemote, eventArgs);
}
}
I’ve also decided to create a simple command that we can just call from the Sitecore ribbon in order to flush this cache manually, this can help in case something get wrong and to avoid the need of manually flushing the redis cache from Azure.
[Serializable]
public class CleanCacheCommand : Sitecore.Shell.Framework.Commands.Command
{
public override void Execute(Sitecore.Shell.Framework.Commands.CommandContext context)
{
var raiser = new CacheRebuildEventRaiser();
var ev = new CacheRebuildEvent { CacheKey = Constants.ClearAll };
raiser.RaiseEvent(ev);
SheerResponse.Alert("Redis Cache flushed");
}
}
That’s very much it! Let’s see this in action now!
So, to make use of this caching foundation, we just need to inject the ICacheManager and use the GetCachedObject method:
var cacheKey = $"RedisCacheTest-{path}";
return _cacheManager.GetCachedObject(cacheKey, () =>
{
var slowMe = DateTime.Now + TimeSpan.FromSeconds(5);
while (DateTime.Now < slowMe)
{
//This is just an expensive operation...
}
return "/some/url";
});
Please note that at the end the cache key will be generated by: {Site Name}{Database Name}{Language Name}{RedisCacheTest}-{path}.
Let’s check now the Redis Cache Console in Azure, we can run the commandSCAN 0 COUNT 1000 MATCH * to get all keys from the cache:
As you can see the “RedisCacheTest” is there!
Let me take the opportunity to introduce the Redis Cache Visual Code extension, find the details here.
The extension provided a quick and easy way to browse the Redis cache contents,