> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bigdata.com/llms.txt
> Use this file to discover all available pages before exploring further.
# SFTP
When it comes to getting your data into our system, SFTP upload is currently the only supported input method. Itβs simple, secure, and easy to implement even for less
technical data providers. That said, weβre actively working on expanding our input mechanisms - an API-based option is on the roadmap, along with other improvements to
make integration even smoother down the line.
Setting up your SFTP connection is as easy as 1-2-3:
SSH authentication is widely regarded as the most secure and reliable method for accessing an SFTP server.
It uses the Secure Shell (SSH) protocol to establish an encrypted connection, ensuring that data transferred
between the client and server remains private and protected from interception. Unlike basic password authentication,
SSH key-based authentication adds an extra layer of security by requiring a private key on the client side that
matches a public key stored on the server. This eliminates the risk of brute-force attacks or stolen credentials,
as the private key never gets transmitted. Additionally, SSH keys can be further secured with passphrases and are
easier to automate safely in scripts and scheduled tasks, making them ideal for both human and system-level access.
Overall, SSH authentication combines strong encryption, identity verification, and secure automation, making it the
best practice for accessing SFTP servers.
These steps will help you generate a new SSH key pair using your terminal on **Linux**, **macOS**, or **Windows**
We will need only your public key (that is safe to share and helps us set up authentication).
Your private key will also be created (and protected by the passkey) - keep it to yourself and never share it with anyone!
* **macOS**: Open the **Terminal** app
* **Linux**: Use your default terminal
* **Windows**: Use **PowerShell** or **Terminal**
Run this command to see if you already have SSH keys:
```bash theme={null}
ls ~/.ssh
```
Look for files like:
* id\_rsa / id\_rsa.pub
* id\_ed25519 / id\_ed25519.pub
If you find them, proceed to [1.6 View your public key](#view-your-public-key)
If not, continue reading ;)
Use the ssh-key command to generate a new SSH key pair:
```bash theme={null}
ssh-keygen -C "your_email@example.com"
```
Adding [your\_email@example.com](mailto:your_email@example.com) is just the argument for -C, which allows you to specify the comment attached to the generated key.
The comment is simply text appended to the key in your public key file, and is typically used as a label for your key.
The default comment is your username @ the hostname of the system you generate your key on, but it can be any string you wish.
You'll be prompted:
```bash theme={null}
Enter file in which to save the key (/home/youruser/.ssh/id_ed25519):`
```
* Press **Enter** to accept the default - **we recommend this option**
* Or enter a custom path if needed
> β οΈ If the file exists, you'll be asked to overwrite - only do this if you're sure you want to do that!
You'll see:
```bash theme={null}
Enter passphrase (empty for no passphrase):
```
* Type a secure passphrase (**you won't see characters as you type**)
* Press Enter, then confirm
> π A passphrase protects your private key, you will need it to authenticate
At this point, you should have both your public and private keys created. Typically, your private key is the file without the extension (id\_ed25519), while your
public key has .pub as the file extension (id\_ed25519.pub)
Whether you already had it or just created it, let's quickly check if everything went well. Run:
```bash theme={null}
cat ~/.ssh/id_ed25519.pub
```
(in case you changed the key file name, make sure to point to the correct file in your command instead of id\_ed25519.pub)
You'll see something like:
```bash theme={null}
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMF... your_email@example.com
```
You can use this key on servers, services like GitHub or any other place that supports SSH authentication.
For our needs, you would need to share .pub file with us, so we import it on our SFTP server. Keep reading to see how to do that.
We are working on a user friendly way to upload your public key but, until we have it, please send the .pub file (your SSH **public** key) to [data@bigdata.com](mailto:data@bigdata.com).
We will confirm the reception of the key by responding to your email and immediately get started with setting up the SFTP for you.
What we need to do is import your public key and configure your access - it typically takes us one day to get this organized and get back
to you with the connection details. Once you receive that from us, proceed to the next step and **test your SFTP connection**.
Meanwhile, while you wait for our response, you can start working on understanding our Bigdata Document Format (BDDF) and preparing the files.
Go [here](/partner-content-upload/getting-started/quick-start-guide) to get started with creating your first BDDF file.
Once you have received a confirmation from us that your public key is imported and SFTP configured (with connection details) it's time to test your connection.
Note that we will share two separate usernames for real-time and historical data delivery - it doesn't make a difference which one you use to test the
connection, if it works for one, it should work for the other as well.
Below you can find three ways to do this, pick the one that suits you the most:
1. Go to the official [FileZilla website](https://filezilla-project.org)
2. Click on Download FileZilla Client (**not Server**)
3. Choose the version for your operating system (Windows/macOS/Linux)
4. Download the standard version (you donβt need Pro unless you require cloud access features)
5. Run the installer:
* on Windows, run the .exe file.
* on macOS, open the .dmg and drag FileZilla into the Applications folder
* on Linux, use your package manager or download the .tar.bz2 archive.
FileZilla on Windows needs a .ppk key (PuTTY format), not OpenSSH by default but we'll help you convert it.
Please note this is about your **private** key, not the public key that you have shared with us previously.
If you don't have your SSH key pair created, please go [here](#generate-ssh-key) and follow instructions.
1. Download [PuTTYgen](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html)
2. Open PuTTYgen.
3. Click Load, and in the file browser, select:
* File type: All Files
* Navigate to C:\Users\yourname.ssh\id\_ed25519 (or wherever your **private** key is)
4. Click Save private key to export it as a .ppk file.
π You can now use this .ppk file in FileZilla.
1. Open FileZilla
2. Go to File β Site Manager
3. Click New Site and give it a name (e.g., RavenPack SFTP)
4. Set the following fields:
* Protocol: SFTP - SSH File Transfer Protocol
* Host: use the hostname we provided to you (most likely sftp.prod.nvirginia.common.ravenpack.com)
* Port: leave empty or use 22 (default SFTP port)
* Logon Type: key file
* User: Your username (provided by RavenPack - use either real-time or historical data username, both should work equally)
* Key file:
\-- On Windows: browse and select the .ppk file
\-- On macOS/Linux: browse and select your private key (e.g., \~/.ssh/id\_ed25519)
5. Click Connect
If successful, you'll see the remote directory structure on the right side of FileZilla. Congratulations!
Testing by using command line interface could be a bit simpler because you don't need to install any additional software. To do this, follow these steps:
1. Open Terminal
2. Run the following command:
```bash theme={null}
sftp -i ~/.ssh/id_ed25519 your_username@your.sftp.host
```
* replace your\_username with the username provided to you (use either real-time or historical data username, both should work equally)
* replace your.sftp.host with the actual SFTP server address (most likely sftp.prod.nvirginia.common.ravenpack.com)
π‘ On some systems, sftp uses ssh underneath and may not accept the -i flag. If that's the case, use this workaround:
```bash theme={null}
ssh -i ~/.ssh/id_ed25519 your_username@your.sftp.host
```
Then try sftp again, assuming \~/.ssh/config is properly configured.
To simplify future connections, add the following to your \~/.ssh/config:
```bash theme={null}
Host ravenpack-sftp
HostName your.sftp.host
User your_username
IdentityFile ~/.ssh/id_ed25519
```
Now you can just run:
```bash theme={null}
sftp ravenpack-sftp
```
Once connected via SFTP, you can run interactive commands:
View remote files:
```bash theme={null}
ls
```
Upload a file:
```bash theme={null}
put local_file.txt
```
Downaload a file:
```bash theme={null}
get remote_file.txt
```
Exit:
```bash theme={null}
bye
```
Here's a Python script that automates the process of testing an SFTP connection using SSH key authentication with the help of the paramiko library.
Install paramiko library, if you haven't already:
```bash theme={null}
pip install paramiko
```
Save the code from the snipped below in a file named **sftp\_text.py**:
```bash theme={null}
import paramiko
import os
# === Configuration ===
HOST = "your.sftp.host" # Replace with your SFTP server
PORT = 22 # Default SFTP port
USERNAME = "your_username" # Replace with your username
PRIVATE_KEY_PATH = os.path.expanduser("~/.ssh/id_ed25519") # Path to your private key
REMOTE_TEST_FILE = "test_remote.txt"
LOCAL_TEST_FILE = "test_local.txt"
# === Connect and Perform SFTP Actions ===
def sftp_connect():
try:
# Load private key
key = paramiko.Ed25519Key(filename=PRIVATE_KEY_PATH)
# Create Transport and SFTP client
transport = paramiko.Transport((HOST, PORT))
transport.connect(username=USERNAME, pkey=key)
sftp = paramiko.SFTPClient.from_transport(transport)
print(f"β
Connected to {HOST} via SFTP")
# List remote directory
print("π Remote directory listing:")
for filename in sftp.listdir():
print(" -", filename)
# OPTIONAL: Upload a test file
if os.path.exists(LOCAL_TEST_FILE):
print(f"β¬οΈ Uploading '{LOCAL_TEST_FILE}'...")
sftp.put(LOCAL_TEST_FILE, REMOTE_TEST_FILE)
print(f"β
Uploaded as '{REMOTE_TEST_FILE}'")
# OPTIONAL: Download the same file back
print(f"β¬οΈ Downloading '{REMOTE_TEST_FILE}' to 'downloaded_{REMOTE_TEST_FILE}'...")
sftp.get(REMOTE_TEST_FILE, f"downloaded_{REMOTE_TEST_FILE}")
print(f"β
Downloaded successfully")
# Close connection
sftp.close()
transport.close()
print("π Disconnected.")
except FileNotFoundError:
print(f"β Private key not found at {PRIVATE_KEY_PATH}")
except paramiko.AuthenticationException:
print("β Authentication failed. Check your SSH key and username.")
except Exception as e:
print(f"β Connection failed: {e}")
# === Run Script ===
if __name__ == "__main__":
sftp_connect()
```
Edit the configuration at the top of the script:
* HOST (most likely sftp.prod.nvirginia.common.ravenpack.com)
* USERNAME (use the username we provided to you - use either real-time or historical data username, both should work equally)
* PRIVATE\_KEY\_PATH (most likely you will just keep the default one)
Just run the script and, if everything goes well, you will see the status message confirming the connection was established successfully:
```bash theme={null}
python sftp_test.py
```
### Next Steps
All done and ready with the SFTP? Head to [Quick Start Guide](/partner-content-upload/getting-started/quick-start-guide) to start preparing your
first BDDF file.