In this article, I explain how the image tagging feature, provided by the Azure AI Vision service, can generate a list of image tags, associated to an image, when an image is uploaded to Optimizely CMS.
The Azure AI services for Vision, powered by Microsoft’s Florence large foundation model, offers a range of features for image analysis and advanced customisation. One key feature as part of this service is the image tagging feature, which has the capability to generate image tags for a wide array of recognisable objects, living beings, scenery, and actions depicted in images.
The tagging extends beyond the primary subject, encompassing the environment (indoor or outdoor), furniture, tools, plants, animals, accessories, gadgets, and more. In instances where tags may be ambiguous or not widely recognised, the API response includes indications to elucidate the meaning of the tag within the context of a known setting. The image used for the image tagging feature is shown above.
Below is a screenshot, showing a code snippet used to create the list of image tags
Response from the API
Image Analysis- Content Tags Operation has finished :
Image height = 682
Image width = 1024
Model version = 2023-02-01-preview
Image Tags Generated Count = 27
“indoor”, Confidence 0.9981
“wall”, Confidence 0.9916
“interior design”, Confidence 0.9902
“living”, Confidence 0.9607
“couch”, Confidence 0.9523
“floor”, Confidence 0.9394
“den”, Confidence 0.9238
“room”, Confidence 0.9193
“pillow”, Confidence 0.9164
“studio couch”, Confidence 0.9103
“loveseat”, Confidence 0.9007
“cushion”, Confidence 0.8865
“sofa bed”, Confidence 0.8859
“house”, Confidence 0.8759
“curtain”, Confidence 0.8743
“coffee table”, Confidence 0.8701
“molding”, Confidence 0.8688
“window treatment”, Confidence 0.8664
“linens”, Confidence 0.8585
“furniture”, Confidence 0.8576
“table”, Confidence 0.8554
“hearth”, Confidence 0.8460
“living room”, Confidence 0.7905
“window”, Confidence 0.7895
“fireplace”, Confidence 0.7752
“chair”, Confidence 0.6184
“ceiling”, Confidence 0.5874
As demonstrated in the above API response, the image utilised for the image tagging feature has produced 27 tags. Subsequently, these tags are assembled into an IList<string> property for publication in the CMS. The resulting list is shown below.
Anil Patel's Blog 