burp suite – Configuring and automating BurpSuite Enterprise Edition scans

I’m evaluating BurpSuite Enterprise Edition and had a question for anyone who might have experience using it.

Architecturally, BurpSuite EE (hereafter “BSEE“) seems to have the following componentry:

  • A web app; used by SecOps to login and manage/configure/run scans
  • 1+ “Agent machines”; servers where “Agents” are installed and running and communicating back to the web app; apparently each Agent machine can run 1+ Agents
  • Agents, software agents installed on agent machines, responsible for running scans against targets and reporting back to the web app
  • The targets that are scanned by the agents

Assuming I’m correct on my understanding of these (which, if I’m not, please begin by correcting me!!!), I’m wondering what the relationships & cardinalities between all these things are.

Say I have 20 microservices (web services using HTTPS) that I want to scan. Do I install the Agents on the 20 servers for each of these 20 microservices, or do I have dedicated “security scanner servers” that I install Agents on, and then configure those agents to run scan against my 20 microservice servers?

So that’s my main question, but I also had two other smaller concerns, specifically:

  • In the web UI it looks like you can either run scans immediately, ad hoc/on demand, or according to a schedule. But what if you want to integrate the scans into your deployment pipeline, automatically? Is there a way to kick off a scan via command line or API and then fetch the report (HTML, XML, PDF, etc.) from disk?
  • Is it possible for developers to configure scans in Community or Pro Edition, export those scan configurations, and import them into BSEE? How about vice versa?

Thanks in advance for any and all concerns!

command line – Automating the installation of MySQL

I am currently trying to automate the purging of MySQL on Ubuntu. However, I am encountering various problems with that.

Here is what I have:

# Uninstall old MySQL version
sudo systemctl stop mysql
sudo apt-get remove mysql-* -y
sudo apt-get autoremove -y
sudo apt-get autoclean -y
sudo rm -rf /etc/mysql /var/lib/mysql /var/log/mysql

# ...
# Do other stuff.
# ...

# Install MySQL latest, see
# https://gist.github.com/kpietru/a3cb08ee074a4418795a
MYSQL_PASSWORD="root"

export MYSQL_PASSWORD="$MYSQL_PASSWORD"

sudo expect -c '
    spawn apt-get install -y mysql-server

    expect "*password* user:"
    send "$env(MYSQL_PASSWORD)r"

    expect "*password* user:"
    send "$env(MYSQL_PASSWORD)r"

    expect "r"
    send "enterr"

interact'

sudo systemctl unmask mysql.service
sudo service mysql start

mysql --version

This works and my Vagrant box loads just fine. However, when I try to use MySQL like so:

mysql -uroot -proot

I get a socket related error message (see https://stackoverflow.com/questions/11990708/error-cant-connect-to-local-mysql-server-through-socket-var-run-mysqld-mysq). I do not want to fix an error here. I want this to work correctly. Therefore I will not go down the rabbit hole of fixing the socket error.

My first intuition was to use sudo apt-get purge -y mysql-* instead of remove, but then I have to automate the prompt responses with expect. I tried to do that with autoinspect, but the expect script generated does not seem to work.

Can you guys help me? Is there anything else I can try?

Thanks in advance & cheers

import – Need help automating a call to a function from a plugin

I have the Magedelight plugin that does pricing per customer. This is a list that I ideally want to update every 20 minutes, which means I need to automate the process.

I don’t have much Magento backend experience, and I could use help figuring out what function gets called when the import button is pressed.

The way it works is that there’s a button to choose the upload file, and the function gets called when you press “save config”

My end goal is to have this process happen automatically every 15 minutes, and I know I can automate that with Cron, but I don’t know which function to call or how to auto upload and set the file for import

Need help regarding automating this stuff in python

Can you help with automating these stuff .I am not able to understand how to do so in python
Attaching the input file and output file

I need the input file to be converted into output file

input file – input_file
output file -output file

Automating mysql_secure_installation in MariaDB Setup

Lots of people want to automate their system setup, and this is frequently done by either using the provider’s new VM hooks to run a script at setup time or later using something like Ansible. If you are trying to automate MariaDB/MySQL installations, it’s easy enough to install MariaDB (e.g., apt-get -y mariadb-server) but you typically want to run the mysql_secure_installation script afterwards to clean up some of the open doors MySQL comes with.

Unfortunately, it’s an interactive script. Here’s an example:

# mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!

In order to log into MariaDB to secure it, we'll need the current
password for the root user. If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none): 
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.

Set root password? (Y/n) y
New password: 
Re-enter new password: 
Password updated successfully!
Reloading privilege tables..
... Success!


By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.

Remove anonymous users? (Y/n) y
... Success!

Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? (Y/n) y
... Success!

By default, MariaDB comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? (Y/n) y
- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? (Y/n) y
... Success!

Cleaning up...

All done! If you've completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!

It’s tedious to type those answers every single time you install MariaDB. Let’s automate it with a script we can inline into any setup script we have.

mysql_secure_installation does the following:

  • sets the root password
  • removes anonymous users
  • disallows remote root logins
  • removes the ‘test’ database and access to it
  • flushes privileges so changes are immediately effective

There are multiple ways to accomplish unattended mysql_secure_installation.  For example, you could use the Expect program.  Or you write a script that echoes “Y” and answers (with newlines) into mysql_secure_installation.

But I prefer to just do what mysql_secure_installation does via SQL.  Here’s a script that accomplishes that.  Note that we are invoking MySQL with ‘-sfu’ which means

  • -s silent
  • -f keep going if there’s an error
  • -u use the following account, which in this case is root

Also be sure to change the “complex_password” below to a good, secure password.

#!/bin/bash

mysql -sfu root <<EOS
-- set root password
UPDATE mysql.user SET Password=PASSWORD('complex_password') WHERE User='root';
-- delete anonymous users
DELETE FROM mysql.user WHERE User='';
-- delete remote root capabilities
DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1');
-- drop database 'test'
DROP DATABASE IF EXISTS test;
-- also make sure there are lingering permissions to it
DELETE FROM mysql.db WHERE Db='test' OR Db='test\_%';
-- make changes immediately
FLUSH PRIVILEGES;
EOS

raindog308

I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.

testing – Automating performance tests for python code in a CI

My apologies in advance for a question that might seem trivial – I am a mostly solo dev in academic environment and a lot of industry best practices don’t necessarily make it here.

Several of my projects run high-performance numerical computation loops. Per se, a single iteration of the loop is rather fast (~1 sec), but there are a lot of them (10-100k per run). Due to that, the performance of the loop is essential to the performance of the whole application and some minor modifications to it (such as unnecessary array type/shape conversions) can slow it down by a lot (latest optimization pass I performed accelerated it by a factor of 20-ish).

As such, it is critical for me to monitor if any changes to the code I am making are having a performance impact on the core loop- be it immediate or by accumulating the loss of performance over time.

I am using a CI suite and number of tools to run unittest, measure test coverage and automatically build the apidocs. However, so far I have not found anything that would perform performance tests as easily or combine their outputs into a graph-over-time. Looking around, I realized that actually comparing performance on different builds is non trivial, given it can be affected by the hardware running the performance tests as well as the software enabling the code isolation and results collection.

Is there a recommended way to perform performance tests to minimize the effects of hardware? Or at least make sure the values are comparable? Is there a standard way of outputting/ingesting performance test results in Python? For instance something along the lines of pytest --duration? Is there a tool/library that can be plugged into a CI suite to perform performance test on a commit/push to a repository?

What are my options for automating Firefox?

My goal is to hit different shortcuts on my keyboard which then open new Firefox windows and open a list of URLs in tabs inside those windows.

What are all the options I have for doing this, specifically on MacOS?

Automating installing all Windows Updates and restart right now

Question in short

How to automate with Windows built-ins (batch/PowerShell) or freely available tools, prefereably command-line, following steps:

  1. Scan for all currently available updates on WU
  2. Download and install them right now (i.e. without any throttling/scheduling/backgrounding)
  3. Restart if necessary, closing whatever doesn’t closes automatically.

I’d also like something that triggers the same background mechanism behind existing WU in Setting panel, not something that will use separate infrastructure by setting up its own download locations or whatever.

Background

I have a PC that doesn’t have regular restarts or fixed working schedule. WU is running, but all its auto-restart facilities are completely disabled so it won’t get in the way of the work at some absolutely inconvenient time. Sometimes a window of opportunity comes up when I can dedicate time to running a full cycle of updates. To do it manually I need to:

  1. Open WU settings
  2. Run “Check for updates” to reveal fresh updates missing for any reason
  3. Click “Download” and then wait arbitrary long time, because apparently this only schedules actual download and both download and installation are heavily throttled depending on PC load
  4. Wait until process is done and manually restart PC if necessary – trying to use “Update & Restart” from power menu often only installs already downloaded stuff and skips over what is not downloaded
  5. Restart often reveals previously hidden updates that DO NOT depend on what was just installed – Settings’ WU panel just seems to stop displaying new updates after at least one freshly installed requests restart

I’d like to automate all those steps and, if possible, reduce number of restarts.

So far I tried

  1. PSWindowsUpdate 3rd party module for PowerShell, but it seems to display different set of available updates – e.g. it offered me to install Silverlight – and whatever it installs does not appear in “View update history” list.

Example:
Settings' WU vs PSWindowsUpdate difference

  1. usoclient.exe looks promising, but there’s no official documentation for it and from some online examples of people cobbling up some working scripts, it seems there’s no reliable way to detect when all updates are done installing and it is time to reboot.

shell – Bash – Automating creation of test files

I’m in the process of understanding shell scripting – this is my simple code for adding test files for python unittest

I’m looking for best practice pointers, variable naming, and of course comments on structure and readability.

EDIT: Script is called by, for example

./create_tests.sh math tensor_add
dir_name="$1"
file_name="$2"
test_dir="test/${dir_name}"
file_in_dir="${test_dir}/${file_name}.py"

addFile() {
    if (( -f "${file_in_dir}" ))
    then
        echo "File already exists in directory, exiting."
        exit 0
    else
        touch "${file_in_dir}"
    fi
}

if (( -d "${test_dir}" ))
then
    echo "Directory already exists, adding the file to this directory."
    addFile
else
    echo "Directory and __init__.py created."
    mkdir "${test_dir}"
    touch "${test_dir}/__init__.py"
    addFile
fi

Automating bookmarking of GitHub stars to Raindrop.io

I’m a lover of GitHub Stars as a quick way to favorite projects. Unfortunately, I have to use a chrome extension to export them as HTML files once in a while to import them to Raindrop.io. I’ve considered using a bookmarklet, which works, but I’d love to not even think about having to toggle my Bookmarks bar and so on.