Downloading Schedule Reports through an API Call #809

arahman63 · 2022-10-26T14:44:16Z

arahman63
Oct 26, 2022

Is it possible to download schedule reports through an api call? I am currently using falconpy and I am having trouble finding which api call would work. I tried using the falcon.get_download() call from the report execution section of the api call; however, when I feed in the report id it says the report entity couldn't be found. I also tried finding the report execution id mentioned in the docs, but couldn't find it . I only have one scheduled report and I got the report ID from the url when you review the report history. There is a trace ID but I removed from the response for the post.

Here's the link I am looking at: https://www.falconpy.io/Service-Collections/Report-Executions.html#report_executions_download_get

Example of the code use:

Service class example (PEP8 syntax)

from falconpy import ReportExecutions

# Do not hardcode API credentials!
falcon = ReportExecutions(client_id=CLIENT_ID,
                          client_secret=CLIENT_SECRET
                          )

id_list = 'ID HERE'

save_file = "some_file.ext"

response = falcon.get_download(ids=id_list)
open(save_file, 'wb').write(response)

code:

from pprint import pp
import os
import json
from falconpy import SpotlightVulnerabilities
from falconpy import ScheduledReports
from falconpy import ReportExecutions
from falconpy import OAuth2
import pandas as pd


if '__name__' == '__main__':
    falcon_client_id = os.environ.get("CLIENT_ID")
    falcon_client_secret = os.environ.get("SECRET")

authorization = OAuth2(creds={
    'client_id': CLIENT_ID,
    'client_secret': SECRET
}, base_url="https://api.crowdstrike.com")
try:
    # calls the token method which generates a dictionary containing the body and the access token
    token = authorization.token()['body']['access_token']
except:
    token = False
response = authorization.oAuth2AccessToken()

If token:
    falcon = ReportExecutions(
        access_token=token, base_url="https://api.crowdstrike.com")
id = "2fff3b432e12934085ebc2sd"
response = falcon.get_download(id)
pd.set_option('display.max_rows', 1000)
pd.set_option('display.max_columns', 1000)
pd.set_option('display.width', 10000)
# DISPLAYS THE ENTIRETY OF THE COLUMN
pd.set_option('display.max_colwidth', None)

#loads the response into a json file 
with open("resp.json", "w") as f:
    json.dump(response, f, ensure_ascii=False, indent=4)
with open("resp.json") as myFile:
    fileData = json.load(myFile)
    dataFrame = pd.DataFrame.from_dict(fileData)

  pp(dataFrame)

Here's the response I get:

{
    "status_code": 404,
    "headers": {
        "Server": "nginx",
        "Date": "Wed, 26 Oct 2022 13:43:00 GMT",
        "Content-Type": "application/json",
        "Content-Length": "179",
        "Connection": "keep-alive",
        "Content-Encoding": "gzip",
        "Strict-Transport-Security": "max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains",
        "X-Cs-Region": "us-1",
        "X-Cs-Traceid": "",
        "X-Ratelimit-Limit": "6000",
        "X-Ratelimit-Remaining": "5998"
    },
    "body": {
        "meta": {
            "query_time": 0.02317307,
            "powered_by": "reports",
            "trace_id": ""
        },
        "errors": [
            {
                "code": 404,
                "message": "Not found."
            }
        ]
    }
}

Answered by jshcodes

Oct 26, 2022

Hi @arahman63 -

Try this code. You should be able to provide it your scheduled report ID and it will download all successful executions of the report in JSON format. (Named after the execution ID.) This is working in my environment.

Don't forget to update it to add in your credentials. For this example, I'm retrieving them from the os environment.

"""Retrieve the contents of a scheduled report and save it to a file."""
import os
import json
from argparse import ArgumentParser, RawTextHelpFormatter
from falconpy import ReportExecutions

parser = ArgumentParser(description=__doc__, formatter_class=RawTextHelpFormatter)
parser.add_argument("-r", "--report", help="ID of the report to retrieve"

View full answer

arahman63 · 2022-10-26T16:30:47Z

arahman63
Oct 26, 2022
Author

I found something that may help:
falcon = ScheduledReports(
access_token=token, base_url="https://api.crowdstrike.com")
response = falcon.launch(id)

scheduled_reports_launch
Launch scheduled report executions for the provided ID(s).

But im not sure how this would get the download

1 reply

jshcodes Oct 26, 2022
Maintainer

The launch operation will trigger a report execution of the scheduled report in question. (You provide the ID.) So if your report has not run its schedule yet, this would be a way to fire it off from the API. (Since you won't see execution IDs until the report has fired at least once, this is a good method to know about. 😄)

jshcodes · 2022-10-26T16:35:29Z

jshcodes
Oct 26, 2022
Maintainer

Hi @arahman63 -

I'll put some code together to better describe the process and post it to our samples library, but to get you started now, here's a high level summary:

For this discussion, assume these two classes have been imported and authenticated successfully as shown below.

from falconpy import ScheduledReports, ReportExecutions
sched = ScheduledReports(client_id="KEY", client_secret="SECRET")
reps = ReportExecutions(auth_object=sched)

Request for a list of report execution IDs using the Scheduled Report ID:
- sched.query_reports(filter="scheduled_report_id:'{REPORT_ID_GOES_HERE}'")
- The IDs returned in the resources list are your report execution IDs.
Using the execution IDs you retrieved, request specific execution's status with reps.get_reports("{EXECUTION_ID_HERE}"). (You can also pass a list here.)
Once the status is marked done, you should be able to retrieve the report with reps.get_download("{EXECUTION_ID_HERE}")

Let us know if you have any questions! 😄

0 replies

arahman63 · 2022-10-26T17:19:41Z

arahman63
Oct 26, 2022
Author

I made the code more readable, building off what you wrote.

scheduledreports = ScheduledReports(
        access_token=token, base_url="https://api.crowdstrike.com")

    report_executions = ReportExecutions(
        access_token=token, base_url="https://api.crowdstrike.com")

report_execution_ids = scheduledreports.query_reports(filter="scheduled_report_id:'2fff3b432e12934085ebc2sd'")
specified_exec_status = report_executions.get_reports(report_execution_ids)
getReportDownloads = report_executions.get_download(report_execution_ids)

with open("resp.json", "w") as f:
    json.dump(specified_exec_status, f, ensure_ascii=False, indent=4)
with open("resp.json") as myFile:
    fileData = json.load(myFile)
    dataFrame = pd.DataFrame.from_dict(fileData)

    pp(dataFrame)

in the json.dump() I would specify what response variable I would like to print out in a json file. the query reports works I get a 200:

{
    "status_code": 200,
    "headers": {
        "Server": "nginx",
        "Date": "Wed, 26 Oct 2022 17:17:19 GMT",
        "Content-Type": "application/json",
        "Content-Length": "195",
        "Connection": "keep-alive",
        "Content-Encoding": "gzip",
        "Strict-Transport-Security": "max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains",
        "X-Cs-Region": "us-1",
        "X-Cs-Traceid": "",
        "X-Ratelimit-Limit": "6000",
        "X-Ratelimit-Remaining": "5998"
    },
    "body": {
        "meta": {
            "query_time": 0.019471532,
            "pagination": {
                "offset": 0,
                "limit": 100,
                "total": 0
            },
            "powered_by": "reports",
            "trace_id": ""
        },
        "resources": [],
        "errors": []
    }
}

But once I feed in the report_execution_ids into report_executions get method and call it into the json dump thing, it doesn't work and it gives me this response:

{
    "status_code": 404,
    "headers": {
        "Server": "nginx",
        "Date": "Wed, 26 Oct 2022 17:18:59 GMT",
        "Content-Type": "application/json",
        "Content-Length": "225",
        "Connection": "keep-alive",
        "Content-Encoding": "gzip",
        "Strict-Transport-Security": "max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains",
        "X-Cs-Region": "us-1",
        "X-Cs-Traceid": "",
        "X-Ratelimit-Limit": "6000",
        "X-Ratelimit-Remaining": "5996"
    },
    "body": {
        "meta": {
            "query_time": 0.105971571,
            "powered_by": "reports",
            "trace_id": ""
        },
        "resources": [],
        "errors": [
            {
                "code": 404,
                "message": "report_execution not found",
                "id": "status_code"
            },
            {
                "code": 404,
                "message": "report_execution not found",
                "id": "headers"
            },
            {
                "code": 404,
                "message": "report_execution not found",
                "id": "body"
            }
        ]
    }
}

0 replies

jshcodes · 2022-10-26T17:23:29Z

jshcodes
Oct 26, 2022
Maintainer

Try this variation, we need to grab the execution IDs from the resources array in the body branch.

scheduledreports = ScheduledReports(
        access_token=token, base_url="https://api.crowdstrike.com")

    report_executions = ReportExecutions(
        access_token=token, base_url="https://api.crowdstrike.com")

report_execution_ids = scheduledreports.query_reports(
    filter="scheduled_report_id:'2fff3b432e12934085ebc2sd'"
    )["body"].get("resources", [])
specified_exec_status = report_executions.get_reports(report_execution_ids)
getReportDownloads = report_executions.get_download(report_execution_ids)

with open("resp.json", "w") as f:
    json.dump(specified_exec_status, f, ensure_ascii=False, indent=4)
with open("resp.json") as myFile:
    fileData = json.load(myFile)
    dataFrame = pd.DataFrame.from_dict(fileData)

    pp(dataFrame)

0 replies

arahman63 · 2022-10-26T17:23:39Z

arahman63
Oct 26, 2022
Author

scheduledreports.query_reports(filter="scheduled_report_id:'2fff3b432e12934085ebc2sd'")["body"].get("resources", {}) fixes the exec status call, but for the last getReportDownloads call, it gives me a 400 saying no ids were given:

{
    "status_code": 400,
    "headers": {
        "Server": "nginx",
        "Date": "Wed, 26 Oct 2022 17:21:11 GMT",
        "Content-Type": "application/json",
        "Content-Length": "182",
        "Connection": "keep-alive",
        "Content-Encoding": "gzip",
        "Strict-Transport-Security": "max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains",
        "X-Cs-Region": "us-1",
        "X-Cs-Traceid": "",
        "X-Ratelimit-Limit": "6000",
        "X-Ratelimit-Remaining": "5992"
    },
    "body": {
        "meta": {
            "query_time": 0.015345274,
            "powered_by": "reports",
            "trace_id": ""
        },
        "errors": [
            {
                "code": 400,
                "message": "No ids given."
            }
        ]
    }
}

1 reply

jshcodes Oct 26, 2022
Maintainer

Did the result state the report had completed execution? I believe there should be exec IDs returned in that call as well.

Minor tweak to the code above (list vs. dictionary):

scheduledreports.query_reports(filter="scheduled_report_id:'2fff3b432e12934085ebc2sd'")["body"].get("resources", [])

arahman63 · 2022-10-26T17:32:25Z

arahman63
Oct 26, 2022
Author

when i used list instead it gave me empty dataframe
dataframe and columns and index

report_execution_ids = scheduledreports.query_reports(filter="scheduled_report_id:'2fff3b432e12934085ebc2sd'")["body"].get("resources", [])

this is what report_execution_ids gives

Empty DataFrame
Columns: []
Index: []

exec status worked with this query
and gave me:

{
	"status_code": 200,
	"headers": {
		"Server": "nginx",
		"Date": "Wed, 26 Oct 2022 17:28:59 GMT",
		"Content-Type": "application/json",
		"Content-Length": "156",
		"Connection": "keep-alive",
		"Content-Encoding": "gzip",
		"Strict-Transport-Security": "max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains",
		"X-Cs-Region": "us-1",
		"X-Cs-Traceid": "c5388dbb-7f25-4c2c-bf3f-b38093fe5c52",
		"X-Ratelimit-Limit": "6000",
		"X-Ratelimit-Remaining": "5997"
	},
	"body": {
		"meta": {
			"query_time": 0.018988878,
			"powered_by": "reports",
			"trace_id": ""
		},
		"resources": [],
		"errors": []
	}
}

and then for the last call with getReportDownloads it gave me this

{
	"status_code": 400,
	"headers": {
		"Server": "nginx",
		"Date": "Wed, 26 Oct 2022 17:29:53 GMT",
		"Content-Type": "application/json",
		"Content-Length": "183",
		"Connection": "keep-alive",
		"Content-Encoding": "gzip",
		"Strict-Transport-Security": "max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains",
		"X-Cs-Region": "us-1",
		"X-Cs-Traceid": "240d74ff-77f4-4806-8c89-9b622b4dbcfa",
		"X-Ratelimit-Limit": "6000",
		"X-Ratelimit-Remaining": "5995"
	},
	"body": {
		"meta": {
			"query_time": 0.014649593,
			"powered_by": "reports",
			"trace_id": ""
		},
		"errors": [{
			"code": 400,
			"message": "No ids given."
		}]
	}
}

0 replies

arahman63 · 2022-10-26T17:41:47Z

arahman63
Oct 26, 2022
Author

report_execution_ids = scheduledreports.query_reports(filter={'scheduled_report_id':'2fff3b432e12934085ebc2sd'})["body"].get("resources", {})

I thought using a dict instead would work, but the same behavior happens

6 replies

arahman63 Oct 26, 2022
Author

getReportDownloads = report_executions.get_download(report_execution_ids) 
with open("resp.json", "w") as f:
    json.dump(getReportDownloads, f, ensure_ascii=False, indent=4)
with open("resp.json") as myFile:
    fileData = json.load(myFile)
    dataFrame = pd.DataFrame.from_dict(fileData)

    pp(dataFrame)

response:

{ "status_code": 400, "headers": { "Server": "nginx", "Date": "Wed, 26 Oct 2022 17:29:53 GMT", "Content-Type": "application/json", "Content-Length": "183", "Connection": "keep-alive", "Content-Encoding": "gzip", "Strict-Transport-Security": "max-age=15724800; includeSubDomains, max-age=31536000; includeSubDomains", "X-Cs-Region": "us-1", "X-Cs-Traceid": "", "X-Ratelimit-Limit": "6000", "X-Ratelimit-Remaining": "5995" }, "body": { "meta": { "query_time": 0.014649593, "powered_by": "reports", "trace_id": "" }, "errors": [ { "code": 400, "message": "No ids given." } ] } }

arahman63 Oct 26, 2022
Author

did you dump the response into a json the way I did it, or did you do a different thing?

arahman63 Oct 26, 2022
Author

I also noticed my get reports response is not as detailed as yours and doesn't contain the status, type . Essentially the resources is empty for me because the query reports gives me an empty dataframe: columns and indices are empty.

jshcodes Oct 26, 2022
Maintainer

Minor change to the code in my last comment, reposting momentarily.

arahman63 Oct 26, 2022
Author

report_execution_ids = scheduledreports.query_reports(filter=f"scheduled_report_id:{id}")["body"].get("resources", [])
another minor change I added to make it compact with format strings, passed in the id as such^

jshcodes · 2022-10-26T18:27:28Z

jshcodes
Oct 26, 2022
Maintainer

Hi @arahman63 -

Try this code. You should be able to provide it your scheduled report ID and it will download all successful executions of the report in JSON format. (Named after the execution ID.) This is working in my environment.

Don't forget to update it to add in your credentials. For this example, I'm retrieving them from the os environment.

"""Retrieve the contents of a scheduled report and save it to a file."""
import os
import json
from argparse import ArgumentParser, RawTextHelpFormatter
from falconpy import ReportExecutions

parser = ArgumentParser(description=__doc__, formatter_class=RawTextHelpFormatter)
parser.add_argument("-r", "--report", help="ID of the report to retrieve", required=True)

report_id = parser.parse_args().report

falcon = ReportExecutions(client_id=os.getenv("FALCON_CLIENT_ID"),
                          client_secret=os.getenv("FALCON_CLIENT_SECRET")
                          )
print(f"Searching for {report_id}")
# Get execution IDs
execution_id_lookup = falcon.reports_executions_query(filter=f"scheduled_report_id:'{report_id}'")
if not execution_id_lookup["status_code"] == 200:
    raise SystemExit("Unable to retrieve report executions from the CrowdStrike API.")

execution_ids = execution_id_lookup["body"]["resources"]
print(f"Found {len(execution_ids)} executions of this report available.")
# Check status of these IDs
exec_status_lookup = falcon.report_executions_get(execution_ids)
if not exec_status_lookup["status_code"] == 200:
    raise SystemExit("Unable to retrieve execution statuses from the CrowdStrike API.")

SAVED = 0
for exec_status in exec_status_lookup["body"]["resources"]:
    status = exec_status["status"]
    exec_id = exec_status["id"]
    print(f"{exec_id} is {status}")
    if status.upper() == "DONE":
        report_detail = falcon.get_download(exec_id)
        if report_detail:
            SAVED += 1
            with open(f"{exec_id}.rpt", "w", encoding="utf-8") as json_output:
                json.dump(report_detail, json_output)
        else:
            print(f"Unable to retrieve report for execution {exec_id}.")

print(f"Retrieval complete, {SAVED} reports were downloaded.")

8 replies

thekarannayak Nov 7, 2022

Hello @jshcodes

Thank you so much for your response!

I also grabbed the ID from the console, and I was able to get the details of report

arahman63 Nov 15, 2022
Author

Hey were you able to format the csv generated after downloading it? I was thinking the csv generated is a deserialized json, and I may need unserialize it to reformat it and convert back to csv to have a working table.

arahman63 Nov 15, 2022
Author

I think the code could be tweaked to get the last execution because that should be the most up to date report . Executions are based on the spotlight vulnerabilities queries you applied onto the report. It can be simply be from your scheduled reports which you can generate with certain filters by downloading it with the falcon dashboard.

thekarannayak Nov 16, 2022

Hello @arahman63 , I was able to download the csv just by using the sample code as mentioned here

arahman63 Nov 17, 2022
Author

I tweaked the code a bit and got rid of the json dump, and used f.write(getReportDownloads.decode("utf-8")) instead because dumping into json gave me unusable data. And then from there I was able to filter our certain columns of the report I wanted to see.

arahman63 · 2022-10-31T14:31:30Z

arahman63
Oct 31, 2022
Author

Thank you for your help. Works as intended!

0 replies

NSH531 · 2022-11-16T06:09:18Z

NSH531
Nov 16, 2022

Pls show an example בתאריך יום ד׳, 16 בנוב׳ 2022, 0:56, מאת Arifur ***@***.***>:

…

Hey were you able to format the csv generated after downloading it? I was thinking the csv generated is a deserialized json, and I may need unserialize it to reformat it and convert back to csv to have a working table. — Reply to this email directly, view it on GitHub <#809 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNBJWJ6FL4P5YZYP2LRDQ3WIQIK5ANCNFSM6AAAAAARPCNCQA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>

1 reply

arahman63 Nov 30, 2022
Author

Where it says json.dump, change it to this:
with open(f"{the_file}.csv", "w",encoding="utf-8") as f:
f.write(getReportDownloads.decode("utf-8"))
SAVED+=1
print(f"{exec_id} successfully saved to {report_id}{exec_id}.csv")

Downloading Schedule Reports through an API Call #809

Uh oh!

Uh oh!

arahman63 Oct 26, 2022

Replies: 10 comments · 17 replies

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

jshcodes Oct 26, 2022 Maintainer

Uh oh!

Uh oh!

jshcodes Oct 26, 2022 Maintainer

Uh oh!

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

jshcodes Oct 26, 2022 Maintainer

Uh oh!

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

Uh oh!

jshcodes Oct 26, 2022 Maintainer

Uh oh!

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

jshcodes Oct 26, 2022 Maintainer

Uh oh!

arahman63 Oct 26, 2022 Author

Uh oh!

Uh oh!

jshcodes Oct 26, 2022 Maintainer

Uh oh!

Uh oh!

thekarannayak Nov 7, 2022

Uh oh!

arahman63 Nov 15, 2022 Author

Uh oh!

arahman63 Nov 15, 2022 Author

Uh oh!

thekarannayak Nov 16, 2022

Uh oh!

arahman63 Nov 17, 2022 Author

Uh oh!

Uh oh!

arahman63 Oct 31, 2022 Author

Uh oh!

NSH531 Nov 16, 2022

Uh oh!

arahman63 Nov 30, 2022 Author

arahman63
Oct 26, 2022

Replies: 10 comments 17 replies

arahman63
Oct 26, 2022
Author

jshcodes Oct 26, 2022
Maintainer

jshcodes
Oct 26, 2022
Maintainer

arahman63
Oct 26, 2022
Author

jshcodes
Oct 26, 2022
Maintainer

arahman63
Oct 26, 2022
Author

jshcodes Oct 26, 2022
Maintainer

arahman63
Oct 26, 2022
Author

arahman63
Oct 26, 2022
Author

arahman63 Oct 26, 2022
Author

arahman63 Oct 26, 2022
Author

arahman63 Oct 26, 2022
Author

jshcodes Oct 26, 2022
Maintainer

arahman63 Oct 26, 2022
Author

jshcodes
Oct 26, 2022
Maintainer

arahman63 Nov 15, 2022
Author

arahman63 Nov 15, 2022
Author

arahman63 Nov 17, 2022
Author

arahman63
Oct 31, 2022
Author

NSH531
Nov 16, 2022

arahman63 Nov 30, 2022
Author