1.1) How to activate 2FA (two factor) authentication?
Follow these steps to turn on Two-Factor Authentication:
Go to the Security settings in your profile (at the bottom left).
Turn on Two-Factor Authentication.
Enter your password and choose how to receive your OTP code:
Use an authentication app (recommended)
Use your email
After receiving the 6-digit code, enter it to confirm that it's set up correctly, and then save your rescue codes.
When 2FA is enabled, an orange shield icon will appear next to your profile picture.
1.2) How is the perimeter of a campaign defined?
The perimeter targeted by the events of a campaign is defined by the user during its creation. It depends on the selected scenario, which requires one or both of the following parts:
"Network" part: ranges and/or individual IP addresses targeted by the requests issued by the BlackNoise Attack Vector, mainly for discovery scans and initial access searches.
"System" part: physical or virtual server or workstation running a Windows, Linux, or macOS operating system. The machines selected by the user during the creation of the campaign are called Target Systems. The Attack Vector establishes connections with each Target System to effectively execute the campaign events. Credentials are required to connect to these systems.
1.3) What is the meaning of the different types of events?
Type logo
Type name
Description
Add a caption...
Network
Event executed at the network layer.
Typical actions for this type of event are: port scans, bruteforce attempts, enumeration requests for services listening on the network, attempted communication for data exfiltration, etc.
Add a caption...
Windows
Event executed on the Windows operating system.
Typical actions for this type of event are: system reconnaissance, system configuration modification, elevation of privileges, password extraction, bypassing defences, vulnerability exploitation (CVE), binary deployment, etc.
Add a caption...
Linux
Event executed on the Linux operating system.
Typical actions for this type of event are: system reconnaissance, system configuration modification, elevation of privileges, password extraction, bypassing defences, vulnerability exploitation (CVE), binary deployment, etc.
Add a caption...
MacOS
Event executed on the macOS operating system.
Typical actions for this type of event are: system reconnaissance, system configuration modification, elevation of privileges, password extraction, bypassing defences, vulnerability exploitation (CVE), binary deployment, etc.
1.4) What does the severity of events mean?
The severity of an event is rated on 2 levels:
Red = High severity
Yellow = Low severity
This severity highlights the most important technical actions to detect from our point of view. Events with a High severity are highly critical behaviors due to the "noise" generated and the impact of the actions considered.
We recommend maximizing detection and reaction efforts on these major events. It is essential to detect them as quickly and effectively as possible.
1.5) What are the possible detection statuses for events and their meanings?
Status
Status title
Event description
Add a caption...
Unqualified
Status assigned by default, no information on the detection of the event is given.
Add a caption...
Undetected
The event is missed, the attack simulation is not detected (no logs and no alert).
Add a caption...
Logged
The security tools created a technical trace (a log) corresponding to a simulated attack. But no alert was triggered and no reaction was taken. To be valid, logs proofs must at least indicate: the source, the destination, the date and time and if possible the type of action recorded.
Add a caption...
Alerted
The security tools identified the simulated attack. An alert or a notification was issued on one of these tools. To be valid, Alerted detection proofs must at least indicate: the source, the destination, the date & time and the type of threat identified. If applicable, the type of remediation made by the security teams can also be specified in the “reaction” section.
1.6) What are the possible statuses for campaigns and their meanings?
Status
Campaign description
Add a caption...
This is the default status assigned when creating a campaign.
The campaign is ongoing. Attack simulations (events) can be triggered from those available in the BlackNoise repository. Users can then enter information relating to the executed attack simulations (status, detection, reaction, proof, comments).
The results make it possible to obtain a measurement of the effectiveness of the detection, in particular through the cybersecurity score calculated according to the information provided.
These results serve as a calibration if control tests are subsequently planned (see Calibrated status). It is not possible to launch control tests in an In progress campaign because the calibration must be finalized before.
Add a caption...
This status is assigned to a campaign for which calibration has been completed.
A Calibrated campaign makes it possible to launch control tests. These tests are composed of all or part of the calibration campaign events. This allows you to monitor the evolution of detection capabilities.
It is no longer possible to launch new events in a Calibrated campaign. Initial event statuses can no longer be changed. Users can also no longer modify the information concerning the sources of detection, the reactions or the evidence filed. The cybersecurity score of the campaign initially calculated can therefore no longer vary.
Users can find out about the detection statuses of events replayed during these control tests. This makes it possible to measure the improvement or regression of detection means. Adding comments to events that are part of regression testing is still possible.
Add a caption...
This status is assigned to a completed campaign for which no more simulations are run. It is also no longer possible to launch control tests. No information can be added or modified. It is kept in the form of an archive to maintain access to the results obtained.
1.7) What is the meaning of the scenario format?
Format
Description
Add a caption...
The scenario reproduces all or part of a Kill Chain, with a sequence of events associated with different Tactics from MITRE ATT&CK
Add a caption...
The scenario has fewer events, centered on a common tactic or approach, but allows for more varied technical implementations
1.8) What are the possible statuses for Attack Vector and their meanings?
Status
Color
Description
Operational
Add a caption...
The Attack Vector is on and connects to the BlackNoise platform. It can be used to run a simulation.
Non operational
Add a caption...
The Attack Vector cannot be used to run a simulation. Please ensure that:
The Attack Vector is started correctly, including the host machine and Docker for the software version
The required communication flows between the Attack Vector and the BlackNoise platform are open (see 1) Deploy an Attack Vector)
Working
Add a caption...
The Attack Vector is used in a simulation. It runs the planned events for this campaign. It cannot be used at the same time by another campaign.
Deploying
Add a caption...
The Attack Vector setup is happening now. It will be ready in a few minutes.
1.9) What is the method for calculating the score?
The campaign score is the average of the scores obtained for all the events in the campaign. This is a measure of the detection rate of attack simulations carried out by BlackNoise.
BlackNoise rates each event according to 2 criteria:
The efficiency of the detection. The more effective it is, the higher the score (an Alerted event thus earns more points than a Logged event, which itself earns more points than an Undetected event). The score is also higher if it concerns a High criticity event than a Low criticity event, this supports the need to detect the most characteristic malicious actions.
The more information there is on the context of the detection, the higher the score
The score is calculated as follows:
Add a caption...
The sum of the scores for all events is reported out of 100 according to the maximum score achievable for all events in this campaign. This maximum score corresponds to the case where all the events are in Alerted status and for which all the technical information is provided (the maximum unit score for a High event is 12.5 points, the maximum unit score for a Low event is 8.75 points). The score of the campaign is therefore calculated as follows:
sum of events points * 100 / (nb of High events * 12.5 + nb of Low events * 8.75)
A score is also calculated for each of the 3 phases of the BlackNoise simplified Kill Chain (3 steps: Initial access & discovery, Compromise & lateral movement, Impact & exfiltration)
The overall score (in the Risks analysis page) is the average of the scores obtained for all the events in all the campaigns, whatever their status.
1.10) Why is my campaign score zero?
By default, all events executed as part of the campaign do not have an assigned detection status. They are in an Undefined state. Enriching the detection information regarding the events allows for qualifying the effectiveness of the defense.
It is therefore necessary to first specify the detection status of the executed events (see below in this FAQ for details about events statuses). This can be done directly from the list of events of the campaign (see Simulations) or on the details page of each event (see Event details (executions, recommendations & comments)).
According to the detection status, it is then possible to provide additional data such as:
Date and time of detection
Detection evidence (you can upload files such as screenshot of an interface, an email or a log file)
Action taken or final status (reaction)
Date and time or reaction
Reaction evidence
The proofs allow attesting to the declared elements and can be used to certify the obtained score.
The more events you qualify and the richer the qualification, the more complete and high the calculated score and indicators will be.
1.11) What is the meaning of the feedback messages during connection tests to a Target System?
To validate the implementation of prerequisites, it is possible to test the connection to a Target System during its creation and at any time from the Resources > Target Systems page. This test is also enabled by default during the campaign creation process.
Depending on the connection protocol being tested, different technical messages may be displayed in case of an error. The following tables summarize their meaning:
WMI tests
Message
Probable causes
DCOM authentication failed
Target System not started
WMI service not activated on Target System
Access to ports 135, 445, and 60000-61000 filtered between Attack Vector and Target System (if the error message is "Something went wrong, please try again later" only the 60000-61000 ports range for RPC is blocked ; the 2 other are opened)
WMI authentication failed: rpc_s_access_denied
Wrong account and/or password
User without the required permissions (user instead of admin)
WinRM or PSRP service not activated on Target System
Access to ports 5985 filtered between Attack Vector and Target System
the specified credentials were rejected by the server
Wrong account and/or password
User without the required permissions (user instead of admin)
SSH tests
Message
Probable causes
Unable to connect to port 22 on <ip_address>
Target System not started
SSH service not activated on Target System
Access to SSH port filtered between Attack Vector and Target System
Authentication failed
Wrong account and/or password
SSH connection failed
Wrong key
1.12) Events troubleshooting
The detailed logs of event execution are accessible from the event detail page, depending on the login profile used. These logs can be exported for investigation by the BlackNoise technical team.
Our R&D team mainly focuses on the techniques described by MITRE ATT&CK and on published detection rules (like Sigma, Suricata...) to develop new simulation events.
We prioritize detection based on common tactics, techniques, and procedures (TTPs) to mimic attacker behavior, rather than just focusing on individual vulnerabilities. Here’s why:
CVEs or 0-day vulnerabilities often depend on a customer's specific environment, making detection based solely on them less useful.
Resource-heavy processes of constantly creating and updating rules for every new CVE are not efficient.
Common attacker behaviors during different stages of an attack (persistence, defense evasion, credential access...) can improve threat detection, regardless of the specific vulnerability.
We do make exceptions for highly impactful vulnerabilities that pose a serious threat and could bypass standard security measures. In these cases, we prioritize developing specific simulation events to address the risks. For example, if significant vulnerabilities are listed in the KEV (Known Exploited Vulnerabilities) catalog, we will look for ways to detect them since they directly affect many of our customers.
2) BlackNoise Attack Vectors & Target Systems deployment
2.1) Configure an HTTP PROXY to be used by the Attack Vector
You can set up an HTTP proxy to connect the Blacknoise Attack Vector (in your network) to the web app in the cloud. You can do this by adding an option in the Attack Vector deployment command from the web app.
Add these settings in the docker run part of the command:
HTTP_PROXY: IP or HOSTNAME of your proxy for connection to port 80 => http://<ip or hostname>:<port>
HTTPS_PROXY: IP or HOSTNAME of your proxy for connection to port 443 => http://<ip or hostname>:<port>. Not needed if the URL is the same as the previous one.
NO_PROXY: IP or HOSTNAME that shouldn't go through any proxy. Not needed if the URL is the same for HTTP and HTTPS, as this will be filled in automatically.
You can run a diagnostic script to get detailed technical information about your Attack Vector's setup and performance. To do this, run the following command from the host system (Linux machine with Docker) where the Attack Vector (the Docker container) is installed:
You can find the value attack_vector_identifier for this command on the Resources > Attack Vectors page of the BlackNoise Web app, in the Identifier column.
The script gives important technical details about the Attack Vector (like the API key and network setup) and shows the results of various tests (like internet connection and connection to the BlackNoise platform). This information helps figure out why there might be issues using the Attack Vector. It needs to be in Operational status, shown by a green dot, to run attack simulations.
2.3) The Attack Vector installation is blocked due to a TLS connection problem
If you see a tls: failed to verify certificate error when you copy the docker command from the BlackNoise app into the terminal of the Attack Vector host, it likely means a network device is inspecting HTTPS traffic.
Error response from daemon: Get "https://registry.blacknoise.co/v2/": tls: failed to verify certificate: x509: certificate signed by unknown authority
You should turn off that inspection feature or allow those flows for the BlackNoise platform.
2.4) How does the Attack Vector connect to the Target Systems to execute the events?
Protocols
The Attack Vector establishes connections with each Target System to execute the campaign events. Commands, programs, or tools are launched on the Target System to accurately replicate the attackers' behaviors. Several protocols can be used for this:
WMI for Windows
SSH for Linux and macOS
Requirements
In order to establish connections between the Attack Vector and the Target Systems, the following steps are necessary:
Provide connection credentials to each Target System. This information is provided when creating the campaign. The different possible connection methods are:
Windows: WMI connection by local account or domain account
Linux and macOS: SSH connection by login & password or key
Open the necessary flows for the connection according to the protocol used. The required ports must be opened between the Attack Vector and the Target Systems. Rules must be added if a software firewall is installed on the operating system of the Target System or if a network firewall is deployed between these subnets.
When creating a campaign, a connection test from the Attack Vector to each Target Systems will be executed to ensure that the flows are open and that the credentials are correct.
Accounts privileges
The higher the privileges of the account, the more events will actually be executed because some actions require high rights. With a standard account, some events will not be executed, which allows for verifying that a user cannot execute these commands or that an EDR blocks them, for example. With an administrator account, this allows for more advanced detection, for example, in case the commands are not blocked.
Some scenarios require specific accounts and privileges, these prerequisites are specified in the description of the scenarios.
2.5) Need help to enable and test WMI access from BlackNoise Attack Vector to your Target System?
The scripts provided below are compatible with Windows 2016 and later versions.
2.5.1) Enable WMI using BlackNoise script
Purpose
This script can be used to enable WMI on the Target System.
Execution
1) Run the script as admin
The account running the command needs to have admin rights. This can be a local admin account or a domain account with local admin access on this machine.
-z: the zones for which a rule will be opened in the Windows firewall
Upon successful execution, you should receive a return as illustrated below:
Add a caption...
2) Reboot the Target System
3) Ensure that the necessary ports are open between the Target System and the BlackNoise Attack Vector.
RPC: 135/TCP
SMB: 445/TCP
Dynamic range of RPC ports: 60000-61000/TCP; or other specific ports range if provided by the "-p" option for command in 1b).
If you have a non-Windows firewall installed on the Target System OS or a network firewall between those subnets you should open ports described above.
4) Ensure the the following registry key is set to 1:
If you want to test your Target System from your own asset, you can use our script:
Purpose
Test the WMI connection from a Windows computer (from which the script is launched) and a Target System
This script is used to validate that WMI has been activated on the Target System, that the credentials supplied are valid and that the flows required for this connection are open between the PC from which the script is run and the Target System. In order for WMI access to be operational from the BlackNoise Attack Vector, these flows must also be open.
Execution
1) Use the script from a Windows computer to check access to a Target System, within a powershell prompt. Do not run the script from the Target System itself.
powershell -f .\blacknoise_check_wmi-v1.0.ps1 target user password domain
Options target, user and password are mandatory while domain is optional.
Upon successful execution and a valid WMI access on the remote system, the message "Success: Successfully retrieved remote computer name" should be returned. Look at the example below:
If you receive the success message below from the check script but WMI access to the Target System does not work nonetheless, log in to the Target System and check that the following registry key is set to 1:
When Negotiate authentication is used in a workgroup, only the built-in Administrator account can access the service. To allow all accounts in the Administrators group to access the service, set the registry value to "1"
2.6) Need help to enable and test WinRM or PSRP access from BlackNoise Attack Vector to your Target System?
2.6.1) Enable WinRM and PSRP
Purpose
This script can be used to enable WinRM and PSRP on the Target System.
Execution
1) Run the Powershell command as admin
The account running the command needs to have admin rights. This can be a local admin account or a domain account with local admin access on this machine.
Enable-PSRemoting-Force
This command will:
Enable WinRM service
Add WinRM listener on port 5985
Add a Windows Firewall entry on the Target System
Upon successful execution, you should see the following results:
WinRM service type changed successfully.
WinRM service started.
WinRM firewall exception enabled.
Configured LocalAccountTokenFilterPolicy to grant administrative rights remotely to local users.
2) Ensure that the 5985/TCP port is open between the Target System and the BlackNoise Attack Vector.
If you have a non-Windows firewall installed on the Target System OS or a network firewall between those subnets you should open this port.
2.6.2) Test WinRM and PSRP using BlackNoise script
Preferably use the BlackNoise web app to test access to your Target System. To do that, navigate to Resources > Target Systems
- When creating the Target System, enable the Launch function and perform a connection test before finalizing
- Access the detail sheet of a Target System with the Connection Testing function.
If you want to test your Target System from your own asset, you can use our script:
Purpose
Test the WinRM/PSRP connection from a Windows computer (from which the script is launched) and a Target System
This script is used to validate that WinRM/PSRP has been activated on the Target System, that the credentials supplied are valid and that the flows required for this connection are open between the PC from which the script is run and the Target System. In order for WinRM and PSRP access to be operational from the BlackNoise Attack Vector, these flows must also be open.
Execution
1) Run the Powershell script as admin from any Windows system with access to the Target System. Do not run the script from the Target System itself.
You will then see a login screen to enter your account password:
Add a caption...
Upon successful execution and a valid Win/PSRP access on the remote system, the message "SUCCESS: Connection to <ip address> with admin successful." should be returned. Look at the example below:
The WinRM activation command successfully opens the flow on the Domain and Private profiles of the local Windows firewall, but not for the Public profile. If the network interface of the Target System used for the connection with the Attack Vector is attached to this Public profile, you must take one of the following actions:
Change the profile of the interface to link it to the Domain or Private profile
Add a rule in the firewall, or modify existing rules, to also open the flow on the Public profile
If you receive the success message below from the check script but WMI access to the Target System does not work nonetheless, log in to the Target System and check that the following registry key is set to 1:
When Negotiate authentication is used in a workgroup, only the built-in Administrator account can access the service. To allow all accounts in the Administrators group to access the service, set the registry value to "1"