Handling HEIC on the web

Filip Drozd

August 14, 2024

Night mode

HEIC (High Efficiency Image Container) is a file format developed by Apple that became the default image format for iPhones and iPads in 2017.

Files that are using this format typically occupy much less space - they can use as little as half the storage of a JPEG. Apple's transition from JPEG to HEIC was driven in part by more efficient storage capabilities that would support the increasing quality of iPhone cameras.

Alright, cool! Let’s render it on our site 🔥

Yeahhh… You see HEIC, while being highy efficient, is not highly supported in the browser environments. As you can see here none of the browsers besides latest releases of Safari support this format.

Consequently the code snippet above won’t work in most browsers.

Why should I care?

Implementing HEIC support in your web app is a time consuming task that often requires revision to make sure that your solution is compatible with your target users’s browsers. If 99% of them use Android then you can probably prioritise other tickets that have been sitting in your backlog.

Let’s look at some data though. According to Statcounter around 28% of smartphones worldwide are running IOS. Those 28% might want to upload a HEIC image they took with their iPhone to your app.

https://gs.statcounter.com/os-market-share/mobile/worldwide

However, most of the businesses are running locally and are limited to a specific region or a country. If your business is targeting the United States market, the share of iOS users on mobile reaches almost 60%. In that case you can be sure that sooner or later a HEIC image will land in your uploader and there has to be a piece of code that will handle it.

https://gs.statcounter.com/os-market-share/mobile/united-states-of-america

Possible solutions

As you can see the only way to be able to reliably render HEIC in a browser is to convert it into a format that is widely supported. Right now, no one knows if Apple is going to change its licensing and allow other browsers to use HEIC images, and even if it does, waiting for it to happen doesn’t seem like a quick solution.

Here are a few approaches that I’m going to go through so that you can compare and choose the one that suits your case best:

Converting on the front-end
Converting on the back-end
Using third party solutions

Front-end conversion

When searching for HEIC conversion tools you are most likely to stumble upon libheif which is a great open source library written in C. But how do we get to run it with JavaScript in the browser environment?

There is the libheif-js library which lets you directly interact with the precompiled JavaScript libheif or run it using WebAssembly. You can also use it in node.js but I’ll cover that later. However, this is a low level solution that requires you to decode the image yourself and then render it using canvas. You can then preview it with a canvas or convert it into a jpg blob and display it with an img tag. Here is an example of how that can be done. As you can see it’s complicated and requires a deep understanding of each step which might not be ideal if you only want to get the job done without getting into details of how libheif works.

Let’s take a look at the library called heic2any which is using libheif under the hood and makes it super simple to get started with HEIC conversion in the browser. Here’s an example implementation:

‍

We have a file picker that accepts an HEIC image and previews it using the img tag. As you can see it takes a few lines of code to convert the image and generate a URL which we can then preview like any other static image.

I think it’s worth elaborating on one thing here…

Good to know: What is a blob?

You might be asking yourself - what is this whole blob thing in JavaScript? You see it everywhere when dealing with files and sometimes it might not be obvious why it’s being used in the first place.

Blobs allow you to create file-like objects made out of raw data and a specified MIME type. They are stored in the browser’s memory and can be accessed using a generated url. It is a perfect solution for any image manipulation because Blobs allow for storing the image data and then easily referencing it - in the example above by passing it as a src attribute.

This video gives a great introduction to how to create a blob yourself and explains a few related concepts like ArrayBuffers:

And just like that we managed to handle an HEIC image and view it without using any backend. However, keep in mind that converting images on the client is not an ideal solution and has its drawbacks that shouldn’t be undermined.

First of all, it’s heavy for the browser to first load the heic2any (2.7 MB) and then to run the actual image conversion. You never know if someone is using your app on the latest MacBook Pro or on the iPhone SE from 2016.

Besides that, you should never trust the client and handing over such crucial task as converting files to the client might be dangerous.

In my opinion frontend conversion should be only used for previewing and if you want to upload a file to the server you should send it in the original HEIC format to continue processing on the backend.

Backend conversion

Depending on your stack there are many tools you can use for handling the HEIC conversion on the backend.

I mentioned libheif before which has dedicated libraries for various languages like Node.js, C#, Python, Rust etc. Let’s take a look at an example implementation in Node.js, specifically using the heic-convert package which is an official user friendly wrapper for libheif.

‍

Using Express.js you can quickly create a server that will handle an HEIC file and return the converted jpeg file. Multer is a middleware that lets you easily handle form data and files - I’m using the memory storage here which doesn’t save any file to the disk but with Multer you can also make the files get saved and return the image url that can be accessed multiple times.

To take it one step further in a production environment you would probably want to save those files to S3 or other storage solution and host them using a CDN.

In this example you simply get the image data which on the client side can be easily parsed to a blob and rendered. To use our backend implementation in the frontend demo I showed above you simply need to replace the heic2any call with a POST request to the server:

Here you go!

‍

We have moved the crucial process of converting an image to the backend making it more reliable. Compared to the frontend conversion the performance will be more consistent as it’s always your server that is running the conversion.

The biggest disadvantage here is the effort that you have to put into spinning up your own server and maintaining it (unless you already have it).

Overall, if you need to reliably handle HEIC conversion with a prospect of also handling storage and referencing those images in a database, implementing a backend conversion is probably the best way to go.

Third party solutions

While implementing the file conversion yourself is not that hard nowadays due to a lot of libraries that get the job done for you, it can still be tricky to choose the right one and be sure that your custom implementation will work well in all the cases.

For example if your output image seems to have a slightly different color palette, you either have to tweak the parameters to match your expected output (assuming the library you’re using implements such interface) or switch to another library that might or might not solve the issue.

Also, the solutions I showed in the previous sections do not handle the file metadata. You would need to implement it separately.

You will also need to perform updates of those libraries and test if the update didn’t break anything in your code.

If you don’t want to struggle with all those things and just want to get a solution that is easy to implement and always works, you should look for an API that will outsource this process for you. For example:

They provide a straightforward documentation and you can always reach out to their support if your business has more specific requirements.

There are also more profound, end-to-end solutions for handling images which go beyond just image conversion:

They allow you to upload images, convert them, process them in many different ways like cropping, resizing or applying filters and access using a CDN. You can also specify the storage solution so it doesn’t have to be their servers. Filestack also provides you with an embedded file picker which means you don’t even have to make any requests yourself.

Back to blog