Skip to content

Commit e48ecf5

Browse files
authored
Merge pull request #84 from nicolgit/main
publish new version with backend api
2 parents 9b31f6f + f7d6e50 commit e48ecf5

37 files changed

+1651
-835
lines changed

INSTALL.md

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Install az-firewall-mon in your environment
2+
3+
When installed in your environment, `az-firewall-mon` will deploy the following resources:
4+
5+
![architecture](./images/deployment.png)
6+
7+
Follow these steps to install a private copy of `az-firewall-mon` in your environment:
8+
9+
* Fork the GitHub repository
10+
* Create a GitHub Personal Access Token (PAT)
11+
* Create all Azure resources
12+
* Configure the GitHub Action to deploy both the SPA and the backend API
13+
* Review Environment variables
14+
* Limit access
15+
16+
# Fork the GitHub repository
17+
18+
The first step is to fork the `az-firewall-mon` repository. This allows you to pull down and build the latest changes and updates from the original repository while maintaining your own personal copy.
19+
20+
* Navigate to: <https://github.com/nicolgit/azure-firewall-mon>
21+
* Click **Fork** > **Create a new fork** (top right of the repository)
22+
* Click **Create fork**
23+
24+
> You now have a fork of the `az-firewall-mon` repository. When a new update is available, you can select **Sync fork** to keep your fork up-to-date and trigger a new build.
25+
26+
# Create a GitHub Personal Access Token (PAT)
27+
28+
1. Go to your GitHub account settings
29+
2. Select **Developer settings** > **Personal access tokens** > **Tokens (classic)** (<https://github.com/settings/tokens>)
30+
3. Click **Generate new token** > **Generate new token (classic)**
31+
4. Give your token a name like "Azure Static Web App Deployment"
32+
5. Expiration: `No Expiration`
33+
6. Select the following scopes:
34+
- `repo` (Full control of private repositories)
35+
- `workflow` (Update GitHub Action workflows)
36+
7. Click **Generate token**
37+
8. **Copy your token** (you won't be able to see it again)
38+
39+
# Create all Azure resources
40+
An instance of `az-firewall-mon` consists of:
41+
* 1 Azure Static Web App (standard plan)
42+
* 1 Azure Maps account
43+
* 1 Azure OpenAI account
44+
* 1 Application Insights instance
45+
46+
You can deploy all these resources to your subscription by clicking the button below:
47+
48+
[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fnicolgit%2Fazure-firewall-mon%2Fmain%2Fbicep%2Fsetup.json)
49+
50+
When deploying, fill in the following parameters:
51+
- `staticWebAppName`: Name for your static web app
52+
- `repositoryUrl`: Your GitHub repository URL (e.g., `https://github.com/username/azure-firewall-mon`)
53+
- `repositoryToken`: Your GitHub PAT created in the previous step
54+
- `branch`: Your main branch (typically 'main')
55+
56+
This will create also an action in your repository that builds and deploys the solution to Azure.
57+
58+
Go to `https://github.com/YOUR-GITHUB-ACCOUNT/azure-firewall-mon/actions` to see the deployment status. When deployment is complete, navigate to Azure Portal > Static Web Apps > View app in browser
59+
60+
# Review environment variables
61+
62+
`az-firewall-mon` requires several environment variables to function properly. These variables are configured automatically during deployment. Here's a reference in case you need to change any:
63+
64+
* **APPLICATIONINSIGHTS_CONNECTION_STRING**: Application Insights connection string
65+
66+
Azure Maps settings:
67+
* **ip_api_key**: Azure Maps API key
68+
* **ip_throttling_calls**: '1'
69+
* **ip_throttling_window_milliseconds**: '1000'
70+
71+
With these settings IP API will return a `429` status code if you make more than 1 call to IP API per second (1000 milliseconds)
72+
73+
Azure OpenAI settings:
74+
* **aoai_api_key**: Azure OpenAI key
75+
* **aoai_endpoint**: Azure OpenAI endpoint
76+
* **aoai_deployment**: Azure OpenAI deployment name
77+
* **llm_throttling_calls**: '5'
78+
* **llm_throttling_window_milliseconds**: '60000'
79+
80+
With these settings Chat API will return a `429` status code if you make more than 5 calls per minute (60000 milliseconds)
81+
82+
Angulare application settings:
83+
* **spa_applicationinsights_connection_string**: Application Insights connection string
84+
* **spa_builddate**: build timestamp
85+
* **spa_local_queuelength**: 100000
86+
87+
> **spa_local_queuelength** is the number of log items that are kept client-side in the browser. When the number of log items in the event-hub is greater than this number, only the latest ones are kept and all others are deleted.
88+
89+
# Limit access
90+
After setup is complete, anyone with a valid Microsoft account can access your copy of `az-firewall-mon`. If you want to restrict access, you have several options:
91+
92+
* [Static Web App Private Endpoint](https://learn.microsoft.com/en-us/azure/static-web-apps/private-endpoint): Expose `az-firewall-mon` on a private IP in your virtual network connected via site-to-site VPN or ExpressRoute to your intranet. This makes the tool available only to your company's employees.
93+
94+
* [Static Web App Authorization](https://learn.microsoft.com/en-us/azure/static-web-apps/authentication-authorization): Since `az-firewall-mon` is a Microsoft account-authenticated app, you can configure a list of emails authorized to access it. The file to update, `staticwebapp.config.json`, is located in [./firewall-mon-app/src/assets](./firewall-mon-app/src/assets/staticwebapp.config.json).
95+

README.md

Lines changed: 41 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
1-
2-
31
<div align="center">
42
<img alt="logo" src="images\logo.png" width="72" height="72" style="vertical-align:middle; background-color: DimGray;border-radius: 15%;">
53
</div>
64
<h1 align="center">az-firewall-mon🧑‍🚒</h1>
75

86
<div align="center">
9-
an <i>alternative and opinionable</i> way to access and inspect Azure Firewall logs
7+
an <i>alternative and opinionated</i> way to access and inspect Azure Firewall logs
108
</div>
119

1210
<br/>
@@ -23,9 +21,9 @@
2321

2422
![azure-firewall-mon-app](images/firewall-mon-app.png)
2523

26-
We all know that Microsoft's recommended approach for analysing Azure Firewall logs is to set up a Log Analytics Workspace to collect all the data and use Kusto (KQL) queries to check the results.
24+
We all know that Microsoft's recommended approach for analyzing Azure Firewall logs is to set up a Log Analytics Workspace to collect all the data and use Kusto Query Language (KQL) queries to check the results.
2725

28-
Azure-Firewall-mon focuses more on providing a tool that can answer the simple question "_what is happening right now?_" in an alternative and hopefully practical way: the idea is to provide an approach much more like [Sysinternals Process Monitor](https://learn.microsoft.com/en-us/sysinternals/downloads/procmon) or [Check Point's SmartView/SmartLog](https://sc1.checkpoint.com/documents/R80.40/WebAdminGuides/EN/CP_R80.40_LoggingAndMonitoring_AdminGuide/Topics-LMG/Using-log-view.htm?tocpath=Logging%7C_____2), where there is no KUSTO queries or dashboards that you need to implement first to get working. Still, all events are available as a _log-stream_.
26+
Azure-Firewall-mon focuses more on providing a tool that can answer the simple question "_what is happening right now?_" in an alternative and practical way. The idea is to provide an approach similar to [Sysinternals Process Monitor](https://learn.microsoft.com/en-us/sysinternals/downloads/procmon) or [Check Point's SmartView/SmartLog](https://sc1.checkpoint.com/documents/R80.40/WebAdminGuides/EN/CP_R80.40_LoggingAndMonitoring_AdminGuide/Topics-LMG/Using-log-view.htm?tocpath=Logging%7C_____2), where you don't need to implement KQL queries or dashboards first to get it working. All events are available as a _log-stream_.
2927

3028
The real strength of the tool is the search field available in the top toolbar. To search for an event, simply start typing and the log flow will be automatically filtered according to those parameters.
3129

@@ -35,7 +33,7 @@ The timestamp field displays the event date in UTC or local format. You can filt
3533

3634
![text filter](images/02-time-filtering.png)
3735

38-
Within this tool, only events from the last 24 hours will appear because this is the duration set on the Event Hub Namespace. A longer duration would slow down the tool and not help answer the question "_what is happening right now_" that az-firewall-mon aims to address.
36+
Within this tool, only events from the last 24 hours will appear because this is the duration set on the Event Hub Namespace. A longer duration would slow down the tool and not help answer the question "_what is happening right now?_" that az-firewall-mon aims to address.
3937

4038
As an alternative to full-text search, you can use the **chatGPT mode**: in the top search field, you can enter a request in natural language, and the system will filter the content accordingly.
4139

@@ -52,63 +50,73 @@ Some examples of queries are as follows:
5250

5351
![chatgpt](images/03-chatgpt.gif)
5452

55-
# Setup a connection with your Azure Firewall
53+
# Set up a connection with your Azure Firewall
54+
Azure-Firewall-mon is an open-source [Single Page Application](https://en.wikipedia.org/wiki/Single-page_application) written in [Angular](https://angular.io/) with an [Azure Functions](https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview) backend written in C# .NET.
55+
56+
Here's the current architecture:
57+
58+
![architecture](./images/architecture.png)
59+
60+
To use this app with **YOUR FIREWALL data**, you have 2 options:
5661

57-
![architecture](images/architecture.png)
62+
1. Use the Azure Firewall mon sample deployment available at <https://az-firewall-mon.duckiesfarm.com>
63+
2. Deploy Azure Firewall mon in your environment
5864

59-
Azure-Firewall-mon is an open source, [Single Page Application](https://en.wikipedia.org/wiki/Single-page_application), written in [Angular](https://angular.io/).
65+
The recommended option is number 2, as this way you can be 100% sure your logs are not going outside your environment. I suggest using the public deployment only for testing purposes.
6066

61-
To use this app with **YOUR data**, you must perform the following steps on your Azure Subscription:
67+
> <https://az-firewall-mon.duckiesfarm.com> uses resources from my subscription (Azure Maps API, Azure OpenAI, Azure Static Web App Standard). These resources have a cost, so I am limiting their usage as much as possible. As a result, the tool may be quite slow. In your own deployment, you can dedicate more resources and achieve better performance.
68+
69+
# Use az-firewall-mon sample deployment
70+
To use this version with your data, you must perform the following steps on your Azure Subscription:
6271

6372
1. Create an Azure Event Hub Namespace
6473
2. Create an Azure Event Hub inside the namespace, with a `1-day retention` and `1 partition`
65-
3. Create a Shared Access Policy, with _Listen_ claim
66-
4. Create an Azure Map Account
67-
5. Create an Azure OpenAI Service
68-
6. Go to OpenAI Studio > Deployments > Create a new deployment using as model `gpt-4o version 2024-05-13`
69-
7. Open the Azure Firewall instance you want to monitor, go to Monitoring > Diagnostic Settings > Add Diagnostic Settings:
74+
3. Create a Shared Access Policy, with _Listen_ claim
75+
4. Open the Azure Firewall instance you want to monitor, go to Monitoring > Diagnostic Settings > Add Diagnostic Settings:
7076

7177
- Select _all_ _logs_ and "Stream to Event Hub"
7278
- Select the Event Hub Namespace and Hub created above
7379
- click `SAVE`
7480

75-
Lazy engineers can performs steps from 1 to 6 by clicking the following button [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fnicolgit%2Fazure-firewall-mon%2Fmain%2Fbicep%2Ffirewall-mon-azure-stuff.json) :-)
81+
If you are a lazy engineer, like me, you can perform all these steps by clicking the following button 😊
7682

77-
Now, open <https://az-firewall-mon.duckiesfarm.com/> and do the following:
83+
[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fnicolgit%2Fazure-firewall-mon%2Fmain%2Fbicep%2Ffirewall-mon-azure-stuff.json)
84+
85+
Open the Azure Firewall instance you want to monitor from the Azure portal, go to Monitoring > Diagnostic Settings > Add Diagnostic Settings:
7886

79-
1. copy in the `Event Hub Connection String` field the connection string of the Shared Access Policy created above
80-
2. copy the corresponding `Event Hub Consumer Group` Name
81-
3. copy in the `Azure Map Account Shared Key` field the primary or secondary Shared Key of the Azure Map Account created above
82-
4. copy in the `Azure OpenAi Endpoint` field the enpoint URI for the OpenAI resouce created above
83-
5. copy in the `Azure OpenAI deployment` field tne name of the deployment created above
84-
6. copy in the `Azure OpenAI access key` field the primary or secondary Shared Key of the Azure OpenAI account created above
85-
7. click on `Let's begin`.
87+
* Select all logs and "Stream to Event Hub"
88+
* Select the Event Hub Namespace and Hub created above
89+
* Click SAVE
90+
91+
Now, open <https://az-firewall-mon.duckiesfarm.com/> and do the following:
8692

87-
# Install Azure-firewall-mon in your environment
93+
1. Copy the connection string of the Shared Access Policy created above into the `Event Hub Connection String` field
94+
2. Copy the corresponding `Event Hub Consumer Group` name
95+
3. Click on `Let's begin`.
8896

89-
[@lukemurraynz](https://github.com/lukemurraynz) has written a very detailed blog post on how deploy Azure-Firewall-mon in an Azure Static Web App. If you prefer this approach, have a look at his blog post <https://luke.geek.nz/azure/deploy-azure-firewall-mon-to-a-static-web-app/>
97+
# Install az-firewall-mon in your environment
9098

91-
> NOTE: `environment.prod.ts` must be updated with your environment information. az-firewall-mon requires an [Application Insights](https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview) instance to work properly.
99+
To install az-firewall-mon in your environment, follow [this guide](INSTALL.md). Once the instance is ready and working, you can go back and follow the instructions in the [Use az-firewall-mon sample deployment](#use-az-firewall-mon-sample-deployment) section. Just replace the URL with the one from your deployment.
92100

93101
# More Information
94102

95-
[Azure Firewall](https://learn.microsoft.com/en-us/azure/firewall/overview) (AF) is a cloud-native and intelligent network firewall security service that provides the best of breed threat protection for your cloud workloads running in Azure. It's a fully stateful, firewall as a service with built-in high availability and unrestricted cloud scalability. It provides both east-west and north-south traffic inspection.
103+
[Azure Firewall](https://learn.microsoft.com/en-us/azure/firewall/overview) (AF) is a cloud-native and intelligent network firewall security service that provides best-of-breed threat protection for your cloud workloads running in Azure. It's a fully stateful, firewall-as-a-service with built-in high availability and unrestricted cloud scalability. It provides both east-west and north-south traffic inspection.
96104

97105
[Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/overview) helps you maximize the availability and performance of your applications and services. It delivers a comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments.
98106

99-
AF (Azure-Firewall-Mon) is integrated with Azure Monitor. This means you can forward AF metrics and logs to:
107+
Azure Firewall is integrated with Azure Monitor. This means you can forward Azure Firewall metrics and logs to:
100108

101109
* Log Analytics Workspace
102110
* Azure Storage
103-
* Event hub
111+
* Event Hub
104112

105113
A [Log Analytics workspace](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-workspace-overview) is a unique environment for log data from Azure Monitor and other Azure services. Each workspace has its own data repository and configuration but might combine data from multiple services.
106114

107-
Be mindful, that the ingest of logs into a Log Analytics workspace has some Latency, so you may see a delay with the logs displaying.
115+
Be mindful that the ingestion of logs into a Log Analytics workspace has some latency, so you may see a delay before logs are displayed.
108116

109-
Latency refers to the time that data is created on the monitored system and the time that it comes available for analysis in Azure Monitor.
117+
Latency refers to the time between when data is created on the monitored system and when it becomes available for analysis in Azure Monitor.
110118

111-
The [Kusto](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/) Query Language is a tool to explore your data in a Log Analytics Workspace. The query uses schema entities that are organized in a hierarchy similar to SQL's: databases, tables, and columns.
119+
The [Kusto Query Language](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/) (KQL) is a tool to explore your data in a Log Analytics Workspace. The query uses schema entities that are organized in a hierarchy similar to SQL's: databases, tables, and columns.
112120

113121
# UIs and tools that inspired Az-Firewall-mon
114122

bicep/firewall-mon-azure-stuff.bicep

Lines changed: 2 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,8 @@
1-
param namespace string = 'fwmonns354526'
1+
@description('Namespace for the Event Hub')
2+
param namespace string = 'fwmonns${uniqueString(resourceGroup().id, deployment().name)}'
23
param hubname string = 'fwmonhub'
34
param sharedkey string = 'fwmonkey'
4-
param mapAccountName string = 'fwmonflags'
5-
param openAiAccountName string = 'fwmonaoai'
65
param location string = resourceGroup().location
7-
param locationaoai string = 'swedencentral'
8-
param fwmonappinsights string = 'fwmonappinsights'
96

107
resource eventHubNamespace 'Microsoft.EventHub/namespaces@2017-04-01' = {
118
name: namespace
@@ -37,49 +34,4 @@ resource firewallMonHub 'Microsoft.EventHub/namespaces/eventhubs/authorizationRu
3734
}
3835
}
3936

40-
resource mapsAccount 'Microsoft.Maps/accounts@2023-06-01' = {
41-
name: mapAccountName
42-
location: location
43-
sku: {
44-
name: 'G2'
45-
}
46-
kind: 'Gen2'
47-
}
48-
49-
resource openAiService 'Microsoft.CognitiveServices/accounts@2022-03-01' = {
50-
name: openAiAccountName
51-
location: locationaoai
52-
sku: {
53-
name: 'S0'
54-
}
55-
kind: 'OpenAI'
56-
properties: {
57-
customSubDomainName: openAiAccountName
58-
networkAcls: {
59-
defaultAction: 'Allow'
60-
virtualNetworkRules: []
61-
ipRules: []
62-
}
63-
publicNetworkAccess: 'Enabled'
64-
}
65-
}
66-
67-
resource cognitiveServicesDeployment 'Microsoft.CognitiveServices/accounts/deployments@2024-04-01-preview' = {
68-
parent: openAiService
69-
name: 'mygpt4'
70-
sku: {
71-
name: 'Standard'
72-
capacity: 2
73-
}
74-
properties: {
75-
model: {
76-
format: 'OpenAI'
77-
name: 'gpt-4o'
78-
version: '2024-05-13'
79-
}
80-
versionUpgradeOption: 'OnceNewDefaultVersionAvailable'
81-
currentCapacity: 2
82-
raiPolicyName: 'Microsoft.Default'
83-
}
84-
}
8537

0 commit comments

Comments
 (0)