Skip to main content

Azure WebApp 500 Errors reporting from AspNetCoreModule

· One min read

Issue Description

Intermittent issues with Azure WebApp constantly stop functioning, a Stop/Start operation brings it back online.

Root Cause

Further investigation using Azure Application Insights, reveals the Azure WebApp was experiencing a few FailedRequestCount, with HTTP 500 Errors. An exception was thrown by a TaskScheduler. Exception of type 'System.OutOfMemoryException' was thrown.

Resolution

In my case, the service that was running on the Azure WebApp was using .NET Core 2.0, the fix was to upgrade to the latest version.

.NET Core 2.0 is an unsupported version and we highly recommend upgrading to the latest version (3.1). Please take a look at this information of the .NET Core official support policy: https://dotnet.microsoft.com/platform/support/policy/dotnet-core

For .NET Core applications I suggest enabling the stdout logs, as those will capture some important errors: https://learn.microsoft.com/en-us/aspnet/core/test/troubleshoot-azure-iis?view=aspnetcore-2.2#aspnet-core-module-stdout-log-azure-app-service-1

If those OutOfMemory exceptions come with a 5xx status code, I would suggest as well using the AutoHeal feature as it will allow setting rules based on that status code to capture a Memory Dump, you can check more information here: https://azure.github.io/AppService/2018/09/10/Announcing-the-New-Auto-Healing-Experience-in-App-Service-Diagnostics.html

Allow Azure DevOps Microsoft Hosted Agent to communicate with Azure KeyVault

· 3 min read

It is best practice to lock down Azure resources to be accessible by location and services that is only to what's required and, the Azure Key vault is no exception.

When using Microsoft Hosted Agents in Azure DevOps, you need to make sure that the AzureCloud IPs for the Azure DevOps regions are opened on the Firewall.

In my case, I was in the: AustraliaEast region and needed to identify and add the following 'AzureCloud' Address Ranges to the KeyVault firewall:

  • "name": "AzureCloud.australiaeast",
  • "id": "AzureCloud.australiaeast",
  • "properties": {
  • "changeNumber": 13,
  • "region": "australiaeast",
  • "regionId": 3,
  • "platform": "Azure",
  • "systemService": "",
  • "addressPrefixes": [
  • "13.70.64.0/18",
  • "13.72.224.0/19",
  • "13.73.192.0/20",
  • "13.75.128.0/17",
  • "13.104.211.128/26",
  • "13.105.16.192/26",
  • "13.105.20.128/26",
  • "13.105.52.192/26",
  • "13.105.53.128/26",
  • "20.37.192.0/19",
  • "20.38.112.0/23",
  • "20.40.64.0/20",
  • "20.40.80.0/21",
  • "20.40.120.0/21",
  • "20.40.176.0/20",
  • "20.42.192.0/19",
  • "20.43.96.0/20",
  • "20.47.37.0/24",
  • "20.47.122.0/23",
  • "20.53.32.0/28",
  • "20.53.40.0/21",
  • "20.53.64.0/18",
  • "20.53.128.0/17",
  • "20.58.128.0/18",
  • "20.60.72.0/22",
  • "20.60.182.0/23",
  • "20.70.128.0/17",
  • "20.135.120.0/21",
  • "20.150.66.0/24",
  • "20.150.92.0/24",
  • "20.150.117.0/24",
  • "20.157.44.0/24",
  • "20.188.128.0/17",
  • "20.190.142.0/25",
  • "20.190.167.0/24",
  • "20.191.192.0/18",
  • "20.193.0.0/18",
  • "20.193.64.0/19",
  • "23.101.208.0/20",
  • "40.79.160.0/20",
  • "40.79.211.0/24",
  • "40.82.32.0/22",
  • "40.82.192.0/19",
  • "40.87.208.0/22",
  • "40.90.18.0/28",
  • "40.90.30.0/25",
  • "40.90.130.80/28",
  • "40.90.130.208/28",
  • "40.90.140.32/27",
  • "40.90.142.160/27",
  • "40.90.147.64/27",
  • "40.90.150.0/27",
  • "40.112.37.128/26",
  • "40.126.14.0/25",
  • "40.126.39.0/24",
  • "40.126.224.0/19",
  • "52.108.40.0/23",
  • "52.108.83.0/24",
  • "52.109.112.0/22",
  • "52.111.224.0/24",
  • "52.113.88.0/22",
  • "52.113.103.0/24",
  • "52.114.16.0/22",
  • "52.114.58.0/23",
  • "52.114.192.0/23",
  • "52.115.98.0/24",
  • "52.120.158.0/23",
  • "52.121.108.0/22",
  • "52.143.199.0/24",
  • "52.143.200.0/23",
  • "52.147.0.0/19",
  • "52.156.160.0/19",
  • "52.187.192.0/18",
  • "52.232.136.0/21",
  • "52.232.154.0/24",
  • "52.237.192.0/18",
  • "52.239.130.0/23",
  • "52.239.226.0/24",
  • "52.245.16.0/22",
  • "104.44.90.64/26",
  • "104.44.93.96/27",
  • "104.44.95.48/28",
  • "104.46.29.0/24",
  • "104.46.30.0/23",
  • "104.209.80.0/20",
  • "104.210.64.0/18",
  • "191.238.66.0/23",
  • "191.239.64.0/19",
  • "2603:1010::/46",
  • "2603:1010:5::/48",
  • "2603:1010:6::/48",
  • "2603:1016:1400:60::/59",
  • "2603:1016:2402::/48",
  • "2603:1016:2500:c::/64",
  • "2603:1017:0:60::/59"

You only need to add the IP ranges of the Region that your Azure DevOps instance sits in.

You can find the region that your Azure DevOps instance sits in by following the article below:

You can retrieve the list of Azure IP Ranges and Service Tags from the following Microsoft JSON file:

Note: These IP ranges can change and update, depending on Microsoft removing and adding new datacenter capability, it is always worth rechecking the list if you find you start having problems with intermittent connectivity to check whether new ranges have been added that haven't been whitelisted.

Failed to delete the private endpoint. Error: Call to Microsoft.Storage/storageAccounts failed

· One min read

Issue Description

Failed to delete the private endpoint. Error: Call to Microsoft.Storage/storageAccounts failed

Root Cause

Azure Backup locks the storage account when you configure protection for any file share in the corresponding account. This provides protection against accidental deletion of a storage account with backed-up file shares.

Resolution

In my case, the Storage account I was attempting to remove the Private Endpoint from was an Azure File Sync storage account, that had Azure File Shares that were getting Backuped Up.

  • Found and removed the lock on the storage account
  • Then successfully delete the private endpoint

More info

Generally, it is recommended that keep the lock taken on the storage account by Azure Backup. If you delete the lock, your storage account will be prone to accidental deletion and if it's deleted, you'll lose your snapshots or backups.

https://learn.microsoft.com/en-us/azure/backup/backup-afs#best-practices

https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/lock-resources

The Cloud Frame of Mind

· 7 min read

Note: Warning buzzwords to follow. Yes, it's that type of article. The views expressed are purely my own.

We are at a turning point in history, where technology is enabling us to do more with less and faster, human life expectancy is longer than what it has ever been, and the world has never been as connected - We are in the 4th industrial revolution.

In an effort to continue to make things better, stronger, and faster we have developed technological aids to assist to drive us forward, in areas such as (but not only) Health, space exploration, travel, and business transformation, one could almost call it the evolution of the first hammer or axe as a tool to help us survive and thrive.

Although there are many technologies in play around the world, artificial intelligence, machine learning, virtual and augmented reality, biotechnology, robotics, and the internet of things to name a few – faster computer processing capability and datacenters all connected to each other and built into the fabric of connectivity across the backbone of the internet, is what I am here to talk about today.

Companies such as Microsoft, in just over 11 years have revolutionized the world of Information Technology. The Microsoft Azure ecosystem is not just ‘someone else computer’, it is so much more than that.

Traditional data centers or on-premises equipment use to require specialized knowledge around areas such as Networking and Hardware, every business function needed to have a physical server, taking up space in a specialized air-conditioned room or just sitting under someone’s desk, running critical functions needed by businesses. I am not saying, this is still not the case and that there are no excuses for such implementations (other than running under someone’s desk), what I am saying is that ‘Information Technology was about Information Technology’ – the ‘Information’ portion of that was a bit harder to access than it is today and was a lot more hands-on to drive value.

Cloud is disrupting traditional IT faster than we think. Today, with 80% of businesses deploying or fully embracing the cloud, we have ‘crossed the chasm’ and are in the ‘early majority stage of the adoption curve.

“A ship is safe in harbor, but that's not what ships are for.” - William Shedd

Using and treating the Microsoft Azure ecosystem, like a normal on-premises datacenter in a world where "a kid working in a garage anywhere in the world, can put you out of business" will slowly but surely limit your potential, with the global scale of the Azure platform, its now possible for businesses, charities and similarly minded individuals to have a global and multi-regional presence.

In the world of digital transformation, technology has become the source of competitive differentiation – If you haven’t realized that your company is a technology company, you have already lost.

Information Technology functional requirements have changed from thinking of performance in terms of the central processing unit (CPU), and Random-access memory (RAM), to thinking about user experience (useability), portability, and scalability.

Azure DevOps

Cloud-based thinking is migrating workloads from IaaS (Infrastructure as a Service) to PaaS (Platform as a Service), or from PaaS to SaaS (Software as a Service), instead of you working for the technology, the technology works for you.

Those previous Information Technology professionals that once worked till the early hours of the morning replacing hardware, keeping systems up and running, are now free to automate, simplify and understand how the technology can work for the consumer. They can now finally concentrate on helping you to deliver and concentrate on the Information that is now at your disposal.

Azure Built-in Controls

The perimeter for security is no longer some black box, running in a dark room blinking into the night – it is your identity, your phone, your password (or password less). Security is everyone’s concern and education of security and how to use technologies is just not the ‘IT guys’ responsibility.

While the cloud can bring greater business value and agility, it can also bring new concerns, including cloud sprawl.

With the pace of change brought on by cloud-based digital technology, your business needs to be highly attuned to the capabilities, skills, and processes necessary from a people perspective to stay relevant and competitive.

What can and cannot be achieved has now been limited by people’s imaginations and the way that people work.

With companies now operating at a global stage and remote working, retaining talent has never become so important, the importance of a company mission statement, strategic priorities, and their ‘Why’ - to use the words of Simon Sinek ‘People don't buy what you do, they buy why you do it.’ Has become a lot more important and visible as employees follow vision, leadership, and technology.

Long story short so, what does this all mean?

In a few bullet points – this is some of what comes to mind when I think of having a Cloud frame of mind means:

  • Collaboration across Information Technology professionals (as the enablers) and Business needs have never been so important.
  • Try, Try, Fail and Try again – Experiment!
  • Think outside of the traditional box, into using technology across Cloud ecosystems such as Azure to drive outcomes.
  • Partnerships with Microsoft and other businesses globally are important to learn, adapt and avoid reinventing the wheel.
  • Shift from Captial to Operational expenses, subscription-based modelling and pay for what you use and consume.
  • Enable, Trust and empower employees.
  • Do not aim for perfection before moving forward or you will never get there.
  • Use Analytics, Integration, and Machine Learning engines to help drive data-driven business decisions.
  • Adopt a continuous learning culture.
  • Embrace Chaos
  • Remember that employee Utilisation does not equal maximum throughput.
  • Build what you cannot buy. Buy what you can't live without
  • Log what is useful, monitor what matters, alert on what's actionable.
  • Empower others (ie shift left) while making sure that everything is auditable, standardized.
  • Develop and promote an ‘everything as code’, ‘everything is automated’ mindset.
  • Test and develop roadmaps to get the most out of upcoming Cloud capabilities.
  • Educate employees on Security and the use of technology to get the most out of it.
  • Remember that some of your clients’ employees are not ‘bogged’ down by what they deem as not possible in IT.
  • Take advantage of the variable cost model of the Cloud
  • Design efficient use of resources via such activities as rightsizing (the process of resizing cloud resources to better match the workload requirements), allocating container costs, finding unused storage and compute, and identifying whether spending anomalies are expected.
  • Automate what is trivial, boring, mundane, and belittling.
  • The Cloud can offer cost savings with resiliency, high-availability, and security automation strategies, you just need to take advantage of it.
  • Champion improvements in People/Process and ways of working
  • Using the Cloud, does not always equal cost savings, however the real value is decreased time-to-deployment, increased agility to adapt to changes and opportunies for innovation, security and modernisation.
  • Concentrate and continue on operational improvements, such as Incident and Problem Management
  • Just as you have to have a push of changes, you have to have a pull of changes to keep the environment lean and avoid waste.

And finally, Revisit, Improve and Reinvest... and enjoy the challenge and opportunities that being in the 4th industrial revolution has to offer!

It is surely an exciting time and is only just the beginning...

Cloud Word

Installing RSAT Tools with PowerShell

· 2 min read

Installing the RSAT (Remote Server Administration Tools for Windows 10) tools using PowerShell. This is just a quick article, written purely as an easy reference.

In the age of the cloud and work from anywhere, Windows 10 allows you easily, install the Remote Server Administration Tools using PowerShell, sometimes (like me) you need these tools not to actually use them - but for the PowerShell modules that come with them to work on scripts locally.

Note: This needs to be run from an elevated PowerShell console (ie ran as Administrator). You can check this using the following:

$currentPrincipal = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
$currentPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

If it returns:

  • False - You are not in an elevated PowerShell window and will have to relaunch as Administrator
  • True - You are all set to go and can continue...

To get a list of all the Remote Server Administration tools you can install run the following:

Get-WindowsCapability -Name RSAT.* -Online

The versions as of the time this article was written are:

  • Rsat.ActiveDirectory.DS-LDS.Tools~~~~0.0.1.0
  • Rsat.BitLocker.Recovery.Tools~~~~0.0.1.0
  • Rsat.CertificateServices.Tools~~~~0.0.1.0
  • Rsat.DHCP.Tools~~~~0.0.1.0
  • Rsat.Dns.Tools~~~~0.0.1.0
  • Rsat.FailoverCluster.Management.Tools~~~~0.0.1.0
  • Rsat.FileServices.Tools~~~~0.0.1.0
  • Rsat.GroupPolicy.Management.Tools~~~~0.0.1.0
  • Rsat.IPAM.Client.Tools~~~~0.0.1.0
  • Rsat.LLDP.Tools~~~~0.0.1.0
  • Rsat.NetworkController.Tools~~~~0.0.1.0
  • Rsat.NetworkLoadBalancing.Tools~~~~0.0.1.0
  • Rsat.RemoteAccess.Management.Tools~~~~0.0.1.0
  • Rsat.RemoteDesktop.Services.Tools~~~~0.0.1.0
  • Rsat.ServerManager.Tools~~~~0.0.1.0
  • Rsat.Shielded.VM.Tools~~~~0.0.1.0
  • Rsat.StorageMigrationService.Management.Tools~~~~0.0.1.0
  • Rsat.StorageReplica.Tools~~~~0.0.1.0
  • Rsat.SystemInsights.Management.Tools~~~~0.0.1.0
  • Rsat.VolumeActivation.Tools~~~~0.0.1.0
  • Rsat.WSUS.Tools~~~~0.0.1.0

To install ALL the RSAT Tools run the following:

Get-WindowsCapability -Name RSAT.* -Online | Add-WindowsCapability -Online

To only install ONLY the Active Directory Users & Computers Remote Administration tool run the following command:

Get-WindowsCapability -Name RSAT.ActiveDirectory* -Online | Add-WindowsCapability -Online

To only install ONLY the Group Policy Management Remote Administration tool run the following command:

Get-WindowsCapability -Name RSAT.GroupPolicy* -Online | Add-WindowsCapability -Online