The most watched videos on my youtube channel are related to security. Azure SQL Networking Secrets and Cloud Security: Secure Access from your Applications to Azure SQL explain private endpoints from different points of views: Azure SQL and Function Apps.

The concepts of Private Endpoints apply to most Azure features and even the relation of “external” features (such as Power BI) with Azure.

There is a catch. There is always a catch. Private Endpoints are only half of the solution. Most people ignore the other half.

Private Endpoint: We create them for one object/feature. They ensure inbound communication to the object.

But what about outbound communication? What happens when an Azure PaaS object needs to make an outbound communication?

There is no unique rule. Each object has its own method to deal with it. The only detail in common is that it means redirecting the outbound communication of the object to inside one of our virtual networks.

On the Cloud Security: Secure Access from your Applications to Azure SQL there are examples about this in relation to Azure functions.

The private endpoint controls the inbound, the VNET integration controls the outbound

Luckly, Azure Open AI doesn’t make outbound connections, right?

Of course it does!

When we connect Azure Open AI to Cognitive Services or other kind of data source, Azure Open AI makes an outbound connection to reach it.

Here is the rub: Azure Open AI has no way to redirect the outbound connection to our private network.

Yes, I tested and proved it: If we make Cognitive Services (or Azure AI Search, the name you like the most) private, Azure Open AI fails to access it. This means all services Azure Open AI access need to remain publicly accessible.

Aren’t they protected by keys?

Of course. But you know the concept of security in layers, right? Leaving one service public is removing one of the security layers. Not good at all.

Show us the Solution

The public connection between Azure Open AI and Cognitive Services passes across the Microsoft Network. I’m talking about a piece of Azure network outside our private network, under Microsoft rules.

The Solution: You fill in a form asking Microsoft to create on their network a private endpoint to your cognitive services. They will take 15 days to analyze. Once they do so, you approve the private endpoint on your cognitive services, and it’s done: The connection between Azure Open AI and Cognitive Services becomes private.

Microsoft Managed Subscription is out of our control, it’s the Microsoft Network

It’s not private to your virtual network, it passes through Microsoft Network, but this is not a big deal. You can make Cognitive Services private, and it will work fine.

Is this Serious?

Yes!

It’s documented on this security article

This is the form you need to fill to request the private endpoint.

I hope Microsoft finds another solution soon. What about Log Analytics?

What else could we do?

The other solution is to use Azure ML Prompt Flow.

Using prompt flow, Azure Open AI doesn’t make outbound connections, all the outbound connections are made from prompt flow, and this can be controlled.