Azure on Mark Allison

Connect Logic App to Azure Sql Securely

Mon, 16 Sep 2024 07:27:43 +0100

The Problem

Recently, I was working on a personal project where I needed to export data from my Azure SQL Database into a JSON file roughly every hour. This JSON file would then be used by a static website hosted on Azure Blob Storage. I wanted a secure way to do this without maintaining passwords, so I decided to use Managed Identity to connect my Logic App to the Azure SQL Database.

SQLMI Restore Blob Storage Access Denied

Tue, 08 Aug 2023 11:24:22 +0100

If you are using Azure SQL Managed Instance and encounter an error like this when restoring a database from an Azure Blob Storage container:

Msg 3201, Level 16, State 2, Line 1

Cannot open backup device 'https://{storage-account}.blob.core.windows.net/my-container/mydb/FULL/mydb-backup.bak'. Operating system error 5(Access is denied.).

You can try the following steps to fix it:

1. Check the SAS token and generate a new one if needed.

The most common cause of this error is that the SAS token used to access the Azure Blob Storage container has expired or is invalid. A SAS token is a shared access signature that grants limited access to a resource in Azure. You can generate a SAS token for your Azure Blob Storage container using the Azure portal, PowerShell, CLI, or SDK.

Which ADF Integration Runtime for my workload?

Wed, 29 Mar 2023 08:26:42 +0000

Self Hosted or Managed Virtual Network for ADF?

I was recently asked which runtime would be better to run ADF pipelines in Azure; use Private Managed Endpoints with the Managed Virtual Network, or provision a Self-Hosted Integration Runtime on a Virtual Machine?

PaaS vs. IaaS

My default position on which technology to use is PaaS where possible. Organisations should focus on providing value to their business, not micro-managing infrastructure. PaaS reduces the operational burden of managing infrastructure in most cases.

How to Move a VM to Another Vnet in Azure

Sun, 19 Mar 2023 09:20:12 +0000

Hello everyone,

In this blog post, I want to share with you a problem that I faced this week with moving a VM from one vnet to another in Azure. It was not as easy as I expected and it required some downtime.

The problem

I had a VM that needed to move to another vnet as part of organisational change.

I thought it would be a simple task of creating a new network interface card (NIC) in the target vnet and attaching it to the VM. However, it seems that it’s not possible to create a NIC in another Vnet and attach it to a VM attached to a different vnet. Also, as you may know a VM must have at least one NIC attached at all times which must be connected to a vnet.

The Importance of Testing Your Azure Backups

Tue, 28 Feb 2023 10:37:04 +0000

My recent experience

Disaster recovery testing is a critical aspect of any organization’s infrastructure. It is crucial to regularly test the resilience of your systems and make sure that they are properly backed up and can be restored in case of an emergency. Recently, a VM that had been restored as part of disaster recovery testing would not start but entered an infinite blue-screen boot loop.

Blue screen loop

Testing Backups

As part of disaster recovery resilience testing, I restored a virtual machine from Azure Recovery Services Vault. The first test was done using the latest snapshot backup, and it took around two minutes to complete. However, when I checked the overview page for the virtual machine in the Azure portal, I noticed that the virtual machine agent status was not ready. This meant that I couldn’t connect to the VM, not even from within the virtual network.

Troubleshooting Get-AzVirtualNetwork

Mon, 16 Jan 2023 12:15:23 +0000

Are you encountering the following error message when working with Azure in PowerShell and running the command Get-AzVirtualNetwork?

'Microsoft.Azure.Management.Network.Models.SecurityRule', 
on 'T MaxInteger[T](System.Collections.Generic.IEnumerable`1[T])' 
violates the constraint of type 'T'.

This error can be caused by an incompatibility between PowerShell 7 and the Az Module and incompatibilty with .NET 7. To fix this issue, try the following steps:

Upgrade to the latest versions of PowerShell 7 and the Az module.
If the issue persists, remove all versions of the Az Module on your system by using the command Uninstall-Module Az -AllVersions.

Sql Virtual Machine Race Conclusion

Thu, 30 Jun 2022 16:28:59 +0100

A couple of months ago I wrote about a race condition in a SQL IaaS VM using the Azure sqlvm resource. I have been working with Microsoft and have come to a better resolution than placing tempdb on a separate remote drive as mentioned in that article.

Suggestion

Microsoft suggested to change the startup type of the SQL Server service to Automatic (Delayed Start). I have now put this into the bicep template for both SQL Server and SQL Server Agent service.

Sql Virtual Machine Race Condition

Tue, 12 Apr 2022 07:54:52 +0000

I’ve been having problems deploying a Sql Virtual Machine in Azure lately and decided to perform some tests to get to the bottom of the issue. I’d like to share some strange behaviour.

Problem

If I deploy a new SQL VM using bicep, it deploys fine. If however, I redeploy the same VM the deployment fails with this error:

Ext_AutomatedBackupError

* code: **Ext_AutomatedBackupError** * Error: **Execution Timeout Expired**. The timeout period elapsed prior to completion of the operation or the server is not responding.;System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out.

In the SQL Server ERRORLOG on the VM:

Where to store Azure Pipelines Variables?

Fri, 04 Mar 2022 07:30:29 +0000

Question

Where should variables be stored for deploying infrastructure-as-code in Azure DevOps pipelines to Azure? Resources need to be deployed to different regions and environments, and attributes for these need to be stored somewhere.

Options

Six options spring to mind.

Parameter files. Seems like the obvious choice.
Template file defaults in azuredeploy.json or main.bicep
Azure DevOps Library variable groups. Seems good, a single place
Azure Pipelines variables in individual pipelines
Configuration file within the project source code
Runtime parameters for setting values at run time

Let’s go through each option and then discuss my preferred approach.

Automate Setting the SqlIaaS Backup Container

Fri, 25 Feb 2022 12:36:08 +0000

Even though Microsoft offer Azure Sql Database and Azure Sql Managed Instance, there are occasions when SQL Server in a VM is required.

The SqlIaaS Azure Resource

If you’ve read my posts before you will know that I deploy resources to the cloud using infrastructure-as-code, with my preferred language being bicep. As a lazy developer, I like to use built-in offerings and PaaS wherever possible. For SQL server on VMs in Azure, the resource Microsoft.SqlVirtualMachine/sqlVirtualMachines is great.

Run DBCC CHECKDB?

Wed, 16 Feb 2022 09:20:58 +0000

As a standard practise for IaaS and on-premises SQL Servers, I usually like to install Ola Hallengren’s maintenance scripts. Why invent the wheel when we already have a super-shiny one? These scripts include integrity checks, which run DBCC CHECKDB.

Is DBCC CHECKDB needed anymore?

It depends. 😉

It comes down to a cost/benefit analysis.

I have been working with Sql Managed Instances quite closely over the past couple of years and wondered whether DBCC CHECKDB needs to be run on these. The whole point of CHECKDB is to notify us that something is wrong but also to do something if the database becomes corrupt in some way. They are very IO intensive operations which can impact user queries on the server.

What-If In Production

Wed, 26 Jan 2022 09:59:06 +0000

What if you could do a dry run of your PowerShell script when making a change to your production environment?

If you’ve read my other posts, you will know I am a big advocate of infrastructure-as-code. Most of my work is done in the Microsoft Azure cloud, and although I encourage source controlling all assets within Azure, sometimes developers create things manually through the portal.

Yes, I know! 🙄

Removing Azure Resources safely

A scheduled drift report should alert developers or DevOps engineers that the production state is out-of-sync with the source controlled code. If this happens, and some resources need removing, then the -WhatIf switch comes in handy and provides confidence to remove items in production that shouldn’t be there.

ADF HTTP/3 Issue

Tue, 30 Nov 2021 07:44:13 +0000

One of my ADF pipelines started failing recently where a Generic REST API Linked Service gave the following error:

525: SSL handshake failed

Digging deeper into the logs, I see a message from Cloudflare.

It appears that the SSL configuration used is not compatible with Cloudflare. This could happen for a several reasons, including no shared cipher suites.

HTTP/3 QUIC

There had been no code changes in ADF, but something on the source side had changed. After working with the data provider, it turns out that Cloudflare was using HTTP/3 which causes the ADF connection to fail.

Grant User JIT Access

Mon, 27 Sep 2021 10:28:47 +0100

An issue came up this week where some external consultants needed access to remote onto an Azure Windows VM to configure a third-party application. The consultants were given AAD logins and are working from home with dynamic IP addresses.

What is the best way to grant them access to RDP onto the VM?

The preferred method would be via a private endpoint into the VNet via point-to-site VPN or Bastion, but external consultants can’t be granted access that way.

Connect to SQLMI with Managed Identity

Wed, 11 Aug 2021 15:42:06 +0100

Consider the scenario where some PowerShell scripts need to be scheduled on an Azure virtual machine, which connects to an Azure SQL Managed Instance.

The PowerShell script needs to run on a schedule and must be able to connect to the managed instance securely. You would rather not have to manage passwords.

What’s the best way to do this? You guessed it, Managed Identity. This post will describe two ways of doing it, one using Azure AD with a Managed Identity and one using SQL Authentication and Azure Key Vault using a Service Principal.

SQL Managed Instance Server Audit Fails

Thu, 05 Aug 2021 08:56:42 +0100

I have set up server auditing for some managed instances in the same virtual cluster to record login success and failed events and write them to blob storage.

Downstream, an ADF pipeline reads the events asynchronously and inserts them into a database for further analysis and reporting.

Server audit stops writing

Recently the server audit stopped writing to blob storage, even though the server audit status was still running and enabled. I have seen failures before where the server audit failed because of an expiring SAS token, but in this case the SAS token has a long expiry date, so know it’s not that.

Bicep Resource Not Found

Thu, 22 Jul 2021 10:02:11 +0100

Bicep stopped deploying with this error

I’m trying to deploy a resource group which has two storage accounts in it with this PowerShell command using a bicep template:

New-AzResourceGroupDeployment -ResourceGroupName $resourcegroupName -TemplateFile .\azuredeploy.bicep -TemplateParameterFile .\azuredeploy.test.parameters.json

Error

New-AzResourceGroupDeployment: Cannot retrieve the dynamic parameters for the cmdlet. Build succeeded: 0 Warning(s), 0 Error(s)

I haven’t made any code changes, just not deployed this resource for a while. I decided to update bicep to the latest version (at the time of writing is Bicep CLI version 0.4.412 (f1169d063e)).

Minimum data file size on MI is 128 GB!

Mon, 10 May 2021 17:15:11 +0100

Storage underlying a Managed Instance

Microsoft Azure should not require you to have deep understanding of the underlying technologies used to provide the service, after all the whole point of PaaS is to abstract you away from implementation details. One of these details is storage of data files in the file system.

When provisioning a General Tier Sql Managed Instance, the maximum storage size for the instance is capped at 8 TB. However this is not the whole story and it’s important to understand some of what is happening under the covers, especially if you are migrating an on-premises SQL Server instance to the cloud that contains a lot of databases or uses partitioning.

Deployments with Bicep and Azure DevOps

Sun, 11 Apr 2021 21:08:20 +0100

Goodbye JSON

Since Microsoft released bicep version 0.3 I thought I’d give it a try because anyone that’s authored ARM templates will know, they tend to make your eyes bleed after a while.

Bicep aims to drastically simplify the authoring experience with a cleaner syntax, improved type safety, and better support for modularity and code re-use.

I’m sold!

I reverse-engineered an existing simple template that deploys two storage accounts by using bicep decompile and got a template like this:

⚠ Careful with Maintenance Window

Fri, 19 Mar 2021 13:41:54 +0000

Maintenance Window now in Preview

(at the time of writing)

On 2nd March 2021, Microsoft announced a long-awaited feature, at least by me, to give choice over when OS and SQL patching occurs behind-the-scenes in Azure SQL Managed Instance. Before this feature, Microsoft would patch the Virtual Cluster supporting Managed Instances outside of regular business hours in the region your Managed Instance is located.

You now get two additional options:

Weekday window, 10PM to 6AM local time Monday – Thursday
Weekend window, 10PM to 6AM local time Friday - Sunday

System default:

EXECUTE denied on Managed Instance

Wed, 17 Mar 2021 12:20:04 +0000

The problem

I came across an interesting (!) issue with an Azure Sql Managed Instance today. One of our users came across this error when trying to SELECT from a view in a database where they are not a member of the db_owner database role or any elevated server-level permission by right clicking the view in SSMS 18.8 and choosing Select Top 1000 Rows.

Select Top 1000 Rows in SSMS

EXECUTE denied on xp_instance_reg_read in mssqlsystemresource

The full text of the error is The EXECUTE permission was denied on the object 'xp_instance_regread', database 'mssqlsystemresource', schema 'sys'. I verified that this was indeed the case with:

Az.Sql Module won't load

Tue, 16 Mar 2021 18:17:22 +0000

Today, the Az.Sql module would not load on my machine when attempting to manually fail over a SQL Managed Instance with this error:

module could not be loaded

I followed the advice of the error message and tried to import the module to get further information.

module is already loaded

Hmmm :thinking:

I decided to uninstall the Azure PowerShell from my machine using the Windows 10 Settings App. Looks like an old version.

Running Sql Server in an Azure Container Instance

Tue, 24 Oct 2017 15:50:31 +0000

Azure Container Instances are still in Preview and not officially available for Windows yet, which made me smile. It took me a while to figure out how to get this working so I thought I’d share what I’ve found. Containers are great for lightweight testing of code before deployment to production servers because they can be created so quickly and they give the same environment to test in very reliably. Now that Microsoft is offering container instances in Azure it means you don’t have to worry about provisioning and configuring your own docker host/cluster. The options for deploying SQL Server are really getting large now, look at this list below!