updated: 2022-08-21
Microsoft Project
Description: I’ve been been looking for ways to enhance Microsoft O365 traffic that has generally always had to flow through an Enterprise Proxy. Pouring through online documentations to find out what traffic needs to go direct without being inspected.
my git repo for microsoft_pac
Resources
Python
Currently I’m working on a project that will allow sync between all the different services that Microsoft requires. This script will be able to extract information from Micrsoft and build a PAC file that can be digested by coporate Proxies and an addtional list that will allow the ability to sync between firewall devices. Typically working with ASA and Palo Alto.
I’ve been using the Powershell script offered as a guideline as well as documenations that advise what cannot be decyrpted to help Optimize Micrsoft traffic destined to the Internet or an Express Route Connection.
Sample usage of my Python Endpoint Script:
>>> from microsoft_pac.endpoint import MicrosoftPac
>>> pac = MicrosoftPac()
>>> pac.create_pac_file(category_type=3)
This will similiarly extract data as defined in the above article Generate Office 365 PAC Files with PowerShell
. Yet, I added an additional case to be able to review the PAC file contents. As you can see in the below test PAC this generates (I omitted a TenantName); you’ll see that Microsoft adds it’s Login URLs that they prefer you not to decrypt:
// This PAC file will provide proxy config to Microsoft 365 services
// using data from the public web service for all endpoints
function FindProxyForURL(url, host)
{
var direct = "DIRECT";
var proxyServer = "PROXY 10.10.10.10:8080";
if(shExpMatch(host, "r1.res.office365.com") //Default
|| shExpMatch(host, "r3.res.office365.com") //Default
|| shExpMatch(host, "r4.res.office365.com") //Default
|| shExpMatch(host, "*.outlook.com") //Default
|| shExpMatch(host, "*.outlook.office.com") //Default
|| shExpMatch(host, "attachments.office.net") //Default
|| shExpMatch(host, "autodiscover.<tenantName>.onmicrosoft.com") //Default
|| shExpMatch(host, "*.sfbassets.com") //Default
|| shExpMatch(host, "*.urlp.sfbassets.com") //Default
|| shExpMatch(host, "skypemaprdsitus.trafficmanager.net") //Default
|| shExpMatch(host, "*.keydelivery.mediaservices.windows.net") //Default
|| shExpMatch(host, "*.msecnd.net") //Default
|| shExpMatch(host, "*.streaming.mediaservices.windows.net") //Default
|| shExpMatch(host, "ajax.aspnetcdn.com") //Default
|| shExpMatch(host, "mlccdn.blob.core.windows.net") //Default
|| shExpMatch(host, "aka.ms") //Default
|| shExpMatch(host, "amp.azure.net") //Default
|| shExpMatch(host, "*.msedge.net") //Default
|| shExpMatch(host, "compass-ssl.microsoft.com") //Default
|| shExpMatch(host, "*.mstea.ms") //Default
|| shExpMatch(host, "*.secure.skypeassets.com") //Default
|| shExpMatch(host, "mlccdnprod.azureedge.net") //Default
|| shExpMatch(host, "videoplayercdn.osi.office.net") //Default
|| shExpMatch(host, "*.skype.com") //Default
|| shExpMatch(host, "*.wns.windows.com") //Default
|| shExpMatch(host, "admin.onedrive.com") //Default
|| shExpMatch(host, "officeclient.microsoft.com") //Default
|| shExpMatch(host, "g.live.com") //Default
|| shExpMatch(host, "oneclient.sfx.ms") //Default
|| shExpMatch(host, "*.sharepointonline.com") //Default
|| shExpMatch(host, "cdn.sharepointonline.com") //Default
|| shExpMatch(host, "privatecdn.sharepointonline.com") //Default
|| shExpMatch(host, "publiccdn.sharepointonline.com") //Default
|| shExpMatch(host, "spoprod-a.akamaihd.net") //Default
|| shExpMatch(host, "static.sharepointonline.com") //Default
|| shExpMatch(host, "*.svc.ms") //Default
|| shExpMatch(host, "<tenantName>-files.sharepoint.com") //Default
|| shExpMatch(host, "<tenantName>-myfiles.sharepoint.com") //Default
|| shExpMatch(host, "*.cdn.office.net") //Default
|| shExpMatch(host, "contentstorage.osi.office.net") //Default
|| shExpMatch(host, "*.onenote.com") //Default
|| shExpMatch(host, "*cdn.onenote.net") //Default
|| shExpMatch(host, "ajax.aspnetcdn.com") //Default
|| shExpMatch(host, "apis.live.net") //Default
|| shExpMatch(host, "cdn.optimizely.com") //Default
|| shExpMatch(host, "officeapps.live.com") //Default
|| shExpMatch(host, "www.onedrive.com") //Default
|| shExpMatch(host, "*.hip.live.com") //Default
|| shExpMatch(host, "*.microsoftonline.com") //Default
|| shExpMatch(host, "*.microsoftonline-p.com") //Default
|| shExpMatch(host, "*.msauth.net") //Default
|| shExpMatch(host, "*.msauthimages.net") //Default
|| shExpMatch(host, "*.msecnd.net") //Default
|| shExpMatch(host, "*.msftauth.net") //Default
|| shExpMatch(host, "*.msftauthimages.net") //Default
|| shExpMatch(host, "*.phonefactor.net") //Default
|| shExpMatch(host, "enterpriseregistration.windows.net") //Default
|| shExpMatch(host, "management.azure.com") //Default
|| shExpMatch(host, "policykeyservice.dc.ad.msft.net") //Default
|| shExpMatch(host, "suite.office.net") //Default
|| shExpMatch(host, "*.aria.microsoft.com") //Default
|| shExpMatch(host, "*.events.data.microsoft.com") //Default
|| shExpMatch(host, "*.o365weve.com") //Default
|| shExpMatch(host, "amp.azure.net") //Default
|| shExpMatch(host, "appsforoffice.microsoft.com") //Default
|| shExpMatch(host, "assets.onestore.ms") //Default
|| shExpMatch(host, "auth.gfx.ms") //Default
|| shExpMatch(host, "c1.microsoft.com") //Default
|| shExpMatch(host, "contentstorage.osi.office.net") //Default
|| shExpMatch(host, "dgps.support.microsoft.com") //Default
|| shExpMatch(host, "docs.microsoft.com") //Default
|| shExpMatch(host, "msdn.microsoft.com") //Default
|| shExpMatch(host, "platform.linkedin.com") //Default
|| shExpMatch(host, "prod.msocdn.com") //Default
|| shExpMatch(host, "shellprod.msocdn.com") //Default
|| shExpMatch(host, "support.content.office.net") //Default
|| shExpMatch(host, "support.microsoft.com") //Default
|| shExpMatch(host, "technet.microsoft.com") //Default
|| shExpMatch(host, "videocontent.osi.office.net") //Default
|| shExpMatch(host, "videoplayercdn.osi.office.net") //Default
|| shExpMatch(host, "*.office365.com") //Default
|| shExpMatch(host, "*.aadrm.com") //Default
|| shExpMatch(host, "*.azurerms.com") //Default
|| shExpMatch(host, "*.informationprotection.azure.com") //Default
|| shExpMatch(host, "ecn.dev.virtualearth.net") //Default
|| shExpMatch(host, "informationprotection.hosting.portal.azure.net") //Default
|| shExpMatch(host, "o15.officeredir.microsoft.com") //Default
|| shExpMatch(host, "officepreviewredir.microsoft.com") //Default
|| shExpMatch(host, "officeredir.microsoft.com") //Default
|| shExpMatch(host, "r.office.microsoft.com") //Default
|| shExpMatch(host, "ocws.officeapps.live.com") //Default
|| shExpMatch(host, "odc.officeapps.live.com") //Default
|| shExpMatch(host, "roaming.officeapps.live.com") //Default
|| shExpMatch(host, "activation.sls.microsoft.com") //Default
|| shExpMatch(host, "crl.microsoft.com") //Default
|| shExpMatch(host, "office15client.microsoft.com") //Default
|| shExpMatch(host, "officeclient.microsoft.com") //Default
|| shExpMatch(host, "insertmedia.bing.office.net") //Default
|| shExpMatch(host, "go.microsoft.com") //Default
|| shExpMatch(host, "support.office.com") //Default
|| shExpMatch(host, "ajax.aspnetcdn.com") //Default
|| shExpMatch(host, "officecdn.microsoft.com") //Default
|| shExpMatch(host, "officecdn.microsoft.com.edgesuite.net") //Default
|| shExpMatch(host, "*.entrust.net") //Default
|| shExpMatch(host, "*.geotrust.com") //Default
|| shExpMatch(host, "*.omniroot.com") //Default
|| shExpMatch(host, "*.public-trust.com") //Default
|| shExpMatch(host, "*.symcb.com") //Default
|| shExpMatch(host, "*.symcd.com") //Default
|| shExpMatch(host, "*.verisign.com") //Default
|| shExpMatch(host, "*.verisign.net") //Default
|| shExpMatch(host, "apps.identrust.com") //Default
|| shExpMatch(host, "cacerts.digicert.com") //Default
|| shExpMatch(host, "cert.int-x3.letsencrypt.org") //Default
|| shExpMatch(host, "crl.globalsign.com") //Default
|| shExpMatch(host, "crl.globalsign.net") //Default
|| shExpMatch(host, "crl.identrust.com") //Default
|| shExpMatch(host, "crl.microsoft.com") //Default
|| shExpMatch(host, "crl3.digicert.com") //Default
|| shExpMatch(host, "crl4.digicert.com") //Default
|| shExpMatch(host, "isrg.trustid.ocsp.identrust.com") //Default
|| shExpMatch(host, "mscrl.microsoft.com") //Default
|| shExpMatch(host, "ocsp.digicert.com") //Default
|| shExpMatch(host, "ocsp.globalsign.com") //Default
|| shExpMatch(host, "ocsp.msocsp.com") //Default
|| shExpMatch(host, "ocsp2.globalsign.com") //Default
|| shExpMatch(host, "ocspx.digicert.com") //Default
|| shExpMatch(host, "secure.globalsign.com") //Default
|| shExpMatch(host, "www.digicert.com") //Default
|| shExpMatch(host, "www.microsoft.com") //Default
|| shExpMatch(host, "*.config.office.net") //Default
|| shExpMatch(host, "*.manage.microsoft.com") //Default
|| shExpMatch(host, "*.office.com") //Default
|| shExpMatch(host, "cdnprod.myanalytics.microsoft.com") //Default
|| shExpMatch(host, "myanalytics.microsoft.com") //Default
|| shExpMatch(host, "myanalytics-gcc.microsoft.com") //Default
|| shExpMatch(host, "workplaceanalytics.cdn.office.net") //Default
|| shExpMatch(host, "workplaceanalytics.office.com") //Default
|| shExpMatch(host, "*.azure-apim.net") //Default
|| shExpMatch(host, "*.flow.microsoft.com") //Default
|| shExpMatch(host, "*.powerapps.com") //Default
|| shExpMatch(host, "activity.windows.com") //Default
|| shExpMatch(host, "ocsp.int-x3.letsencrypt.org") //Default
// Allow and Optimize
|| shExpMatch(host, "smtp.office365.com") //Allow
|| shExpMatch(host, "*.protection.outlook.com") //Allow
|| shExpMatch(host, "*.mail.protection.outlook.com") //Allow
|| shExpMatch(host, "outlook.office.com") //Optimize
|| shExpMatch(host, "outlook.office365.com") //Optimize
|| shExpMatch(host, "*.lync.com") //Allow
|| shExpMatch(host, "*.teams.microsoft.com") //Allow
|| shExpMatch(host, "teams.microsoft.com") //Allow
|| shExpMatch(host, "*.broadcast.skype.com") //Allow
|| shExpMatch(host, "broadcast.skype.com") //Allow
|| shExpMatch(host, "<tenantName>.sharepoint.com") //Optimize
|| shExpMatch(host, "<tenantName>-my.sharepoint.com") //Optimize
|| shExpMatch(host, "*.officeapps.live.com") //Allow
|| shExpMatch(host, "*.online.office.com") //Allow
|| shExpMatch(host, "office.live.com") //Allow
|| shExpMatch(host, "*.msftidentity.com") //Allow
|| shExpMatch(host, "*.msidentity.com") //Allow
|| shExpMatch(host, "account.activedirectory.windowsazure.com") //Allow
|| shExpMatch(host, "accounts.accesscontrol.windows.net") //Allow
|| shExpMatch(host, "adminwebservice.microsoftonline.com") //Allow
|| shExpMatch(host, "api.passwordreset.microsoftonline.com") //Allow
|| shExpMatch(host, "autologon.microsoftazuread-sso.com") //Allow
|| shExpMatch(host, "becws.microsoftonline.com") //Allow
|| shExpMatch(host, "clientconfig.microsoftonline-p.net") //Allow
|| shExpMatch(host, "companymanager.microsoftonline.com") //Allow
|| shExpMatch(host, "device.login.microsoftonline.com") //Allow
|| shExpMatch(host, "graph.microsoft.com") //Allow
|| shExpMatch(host, "graph.windows.net") //Allow
|| shExpMatch(host, "login.microsoft.com") //Allow
|| shExpMatch(host, "login.microsoftonline.com") //Allow
|| shExpMatch(host, "login.microsoftonline-p.com") //Allow
|| shExpMatch(host, "login.windows.net") //Allow
|| shExpMatch(host, "logincert.microsoftonline.com") //Allow
|| shExpMatch(host, "loginex.microsoftonline.com") //Allow
|| shExpMatch(host, "login-us.microsoftonline.com") //Allow
|| shExpMatch(host, "nexus.microsoftonline-p.com") //Allow
|| shExpMatch(host, "passwordreset.microsoftonline.com") //Allow
|| shExpMatch(host, "provisioningapi.microsoftonline.com") //Allow
|| shExpMatch(host, "*.compliance.microsoft.com") //Allow
|| shExpMatch(host, "*.manage.office.com") //Allow
|| shExpMatch(host, "*.protection.office.com") //Allow
|| shExpMatch(host, "*.security.microsoft.com") //Allow
|| shExpMatch(host, "compliance.microsoft.com") //Allow
|| shExpMatch(host, "manage.office.com") //Allow
|| shExpMatch(host, "protection.office.com") //Allow
|| shExpMatch(host, "security.microsoft.com") //Allow
|| shExpMatch(host, "*.portal.cloudappsecurity.com") //Allow
|| shExpMatch(host, "account.office.net") //Allow
|| shExpMatch(host, "admin.microsoft.com") //Allow
|| shExpMatch(host, "home.office.com") //Allow
|| shExpMatch(host, "portal.office.com") //Allow
|| shExpMatch(host, "www.office.com") //Allow
|| shExpMatch(host, "portal.microsoftonline.com") //Allow
)
{
return direct;
}
return proxyServer;
}
My Python Script
My script works by first checking the latest version and storing that version locally to compare if I need to go out and sync a new PAC or list of Firewall rules that I deploy globally. Since enterprise environments have Proxies or tend to try to decrypt traffic I also offer options to supply an Enterprise CA or simply supply Proxy URL to redirect the request to Microsoft through.
NOTE: See also my info on how to update and add an enterprise certificate to a CA. Enterprise SSL Certs
.
endpoints.py sample
import requests, uuid
from .exceptions import *
"""
see
https://docs.microsoft.com/en-us/archive/blogs/onthewire/new-office-365-url-categories-to-help-you-optimize-the-traffic-which-really-matters
"""
def web_api_get(method, instance, proxy=False, proxy_url=None, client_id=None, verify=True):
"""
Used to retrieve Microsoft Endpoints in JSON format and the
current version of the latest Microsoft Endpoints release.
Args
----
method: str\n
\tSupply method of operation. Valid Options: version, endpoints\n
instance: str\n
\tSupply the instance type you are connecting to O365 for.
Valid Options: Worldwide, USGOVDoD, USGOVGCCHigh, Germany\n
proxy: bool\n
\tDefines if Proxy should be set within the request.
If set to 'True' must supply proxy_url. Default: 'False'\n
proxy_url: str\n
\tUsed if proxy value is set to 'True'. Proxy URL is passed into request.\n
client_id: str\n
\tGUID string used in request to identify your request.
Using standard 128-bit integer number format. Autogenerates if none supplied\n
verify: bool\n
\tBool value to pass if certificate checks should be made.
Can also be used to pass a custom CA see requests documents.
"""
website = 'https://endpoints.office.com'
valid_methods = ['version', 'endpoints']
if method not in valid_methods:
raise ValueError(f"Unsuported method value={method}")
request_path = f"{website}/{method}/{instance}"
if not client_id:
client_id = str(uuid.uuid4())
params = { 'clientrequestid': client_id, 'format': 'json'}
if proxy:
if not proxy_url:
raise ValueError(proxy_url)
else:
proxies = { i: proxy_url for i in ['http', 'https']}
r = requests.get(request_path, proxies=proxies, params=params, verify=verify)
else:
r = requests.get(request_path, params=params, verify=verify)
try:
r.raise_for_status()
return r.json()
except requests.exceptions.SSLError as err:
raise CertificateError
NOTE: Minus the addition of ‘CertificateError’ exception getting raised this code will work on it’s own to supply a JSON response of the current version or the actual response from Microsoft on all the endpoint information.
Sample JSON responses:
>>> import microsoft_pac.endpoint
>>> microsoft_pac.endpoint.web_api_get('version', 'Worldwide')
{'instance': 'Worldwide', 'latest': '2021042900'}
>>>
>>> endpoints = microsoft_pac.endpoint.web_api_get('endpoints', 'Worldwide')
>>> endpoints[0].keys()
dict_keys(['id', 'serviceArea', 'serviceAreaDisplayName', 'urls', 'ips', 'tcpPorts', 'expressRoute', 'category', 'required'])
>>> len(endpoints)
108
>>> endpoints[0]
{'id': 1, 'serviceArea': 'Exchange', 'serviceAreaDisplayName': 'Exchange Online', 'urls': ['outlook.office.com', 'outlook.office365.com'], 'ips': ['13.107.6.152/31', '13.107.18.10/31', '13.107.128.0/22', '23.103.160.0/20', '40.96.0.0/13', '40.104.0.0/15', '52.96.0.0/14', '131.253.33.215/32', '132.245.0.0/16', '150.171.32.0/22', '204.79.197.215/32', '2603:1006::/40', '2603:1016::/36', '2603:1026::/36', '2603:1036::/36', '2603:1046::/36', '2603:1056::/36', '2603:1096::/38', '2603:1096:400::/40', '2603:1096:600::/40', '2603:1096:a00::/39', '2603:1096:c00::/40', '2603:10a6:200::/40', '2603:10a6:400::/40', '2603:10a6:600::/40', '2603:10a6:800::/40', '2603:10d6:200::/40', '2620:1ec:4::152/128', '2620:1ec:4::153/128', '2620:1ec:c::10/128', '2620:1ec:c::11/128', '2620:1ec:d::10/128', '2620:1ec:d::11/128', '2620:1ec:8f0::/46', '2620:1ec:900::/46', '2620:1ec:a92::152/128', '2620:1ec:a92::153/128', '2a01:111:f400::/48'], 'tcpPorts': '80,443', 'expressRoute': True, 'category': 'Optimize', 'required': True}
O365 Restrict Tenant
The problem with just using the data that Microsoft provides is you need to decrypt those Microsoft Login URL to be able to enforce Tenant Restrictions. Therefore an exception needs to be made. Depending on where you are trying to enforce Tenant Restriction you’ll need to ensure to pass the following URLs to your SSL Decryption device that is capable of decrypting and inserting a header into the packet:greg
- login.microsoftonline.com
- login.microsoft.com
- login.windows.net
See your vendor documenation in order how to perform this. Although it is very simple as most any security device built to decrypt traffic can handle this.
The trick is being able to create the proper PAC file and/or Firewall URL lists to following Microsoft’s instructions on what you should not be decrypting while still decrypting what you require, if any.
NOTE: onmicrosoft.com domain should additionally be added into the comma seperated header insertion field: “Restrict-Access-To-Tenants”.
Additional Alternative Restrictions
Additional restriction for Browser Apps requires:
Full code as provided from Microsoft:
// Allows access to the listed tenants.
if (
oSession.HostnameIs("login.microsoftonline.com") ||
oSession.HostnameIs("login.microsoft.com") ||
oSession.HostnameIs("login.windows.net")
)
{
oSession.oRequest["Restrict-Access-To-Tenants"] = "<List of tenant identifiers>";
oSession.oRequest["Restrict-Access-Context"] = "<Your directory ID>";
}
// Blocks access to consumer apps
if (
oSession.HostnameIs("login.live.com")
)
{
oSession.oRequest["sec-Restrict-Tenant-Access-Policy"] = "restrict-msa";
}
The additional login.live.com URL with a required “restrict-msa” in order to prevent online usage of applications outside of a users tenanat. Leveraging the above inside the HTTP Header Insertion policy created above seems to achieve this. By adding the additional URL and setting that value to restrict-msa appears to resolve this issue where Users are able to login through the Web and use Microsoft Web Apps. What that will do is not allow a user to access login.live.com forcing users to access Microsoft Online Services through one of the 3 URL above. Without this additional header insertion users will still be able to leverage O365 servies online.
NOTE: I have done some testing in adding the tenant restrictions in the header on login.live.com instead of using the header insertion of restrict-msa and that did allow login under a personal Microsoft Live account with minimal Web Applicaiton Usage. So, for full block of O365 and restriction to specific tenants the additional HTTP Header Insertion policy would, more than likely, be required.