SFTP Integration

Ask a room of integration architects what the most reliable pattern in their estate is, and after the fashionable answers about event streaming and real-time APIs settle down, someone honest will admit it is the nightly file drop over SFTP. File-based integration refuses to die because it solves a problem the shiny patterns do not: how do two organisations that trust each other just enough to exchange data, but not enough to open a live connection into each other's systems, move information reliably every single day? SFTP is the answer that has held up for twenty years, and it is worth understanding properly. This guide sits inside a broader map of how enterprise systems connect, so if you want the wider context first, start with the enterprise system integration pillar and come back here for the file-transfer detail.

The one thing to get straight up front: SFTP is the SSH File Transfer Protocol. It runs as a subsystem of SSH, the same secure shell you use to administer Linux servers. It is not FTP with a security bolt-on. It is a completely different protocol that happens to do a similar job. Confusing SFTP with FTPS is the single most common mistake I see in integration specifications, and it leads to firewall rules, certificates and client libraries that solve the wrong problem.

1. What SFTP is (the SSH File Transfer Protocol)

SFTP stands for the SSH File Transfer Protocol. The critical word is SSH. When a client opens an SFTP session, it first establishes a normal SSH connection to the server, exactly the same secure channel that a system administrator uses to log in to a remote shell. Once that encrypted, authenticated SSH connection is up, the client asks the server to start the SFTP subsystem, and file operations then flow inside that single protected channel. There is no separate data connection, no cleartext control channel, no negotiation of ports. Everything, the authentication, the directory listings, the file contents and the status responses, travels inside one encrypted SSH stream on one port.

This architecture is why SFTP behaves so differently from the FTP family. FTP was designed in the early 1970s, long before the internet had any concept of hostile networks, and it splits its work across a control connection and one or more separate data connections. SFTP was designed decades later on top of SSH, which was built from the start to assume the network is untrusted. That single design decision, riding on SSH, gives SFTP its encryption, its integrity checking, its authentication options and its firewall-friendliness for free, because it inherits all of them from the SSH layer beneath it.

The protocol itself is a request and response protocol. The client sends packets that say things like "open this file", "read these bytes", "write this block", "list this directory" or "rename this path", and the server responds with data or a status code. Because it is a structured protocol rather than a stream of commands and replies over separate connections, SFTP can do things plain FTP cannot: it can resume interrupted transfers cleanly, check remote file attributes, set permissions, and manipulate the remote filesystem in ways that feel much closer to a real remote filesystem than to a file-copy tool. That richness is part of why it became the default for serious machine-to-machine file exchange.

One point of honesty about the standard: SFTP is defined by a series of IETF SECSH working group drafts for the SSH File Transfer Protocol, and those drafts were never finalised into a single ratified RFC in the way many people assume. The most widely deployed behaviour in the real world is the version implemented by OpenSSH, which is what the overwhelming majority of servers and clients speak. I cover this properly in the References section, but keep it in mind: when someone says "the SFTP standard", the practical reality is "what OpenSSH does", and that is a good thing, because it is one dominant, well-tested implementation rather than a fragmented mess.

2. SFTP versus FTP versus FTPS

This is the comparison that causes the most expensive confusion, so it is worth being precise. There are three protocols people casually lump together, and they are genuinely different animals. Plain FTP is the original File Transfer Protocol with no encryption at all. FTPS is FTP wrapped in a TLS/SSL layer, the same transport security that protects HTTPS. SFTP is the SSH File Transfer Protocol, an entirely separate protocol running over SSH. The names are similar and the jobs overlap, which is exactly why they get mixed up, but underneath they share almost nothing.

Read that table and the practical guidance falls out on its own. Plain FTP has no place in any modern integration, full stop, because it sends the password and the payload across the network in the clear. Between FTPS and SFTP, both are genuinely secure, but SFTP wins on operational simplicity almost every time. FTPS inherits FTP's awkward split between control and data connections, which means firewalls and network-address-translation devices struggle to track it, especially once TLS hides the port numbers they would normally inspect. SFTP uses a single connection on a single port, port 22, so a firewall rule is trivial and the connection behaves predictably through NAT. That single-port simplicity is the quiet reason SFTP won the market for business file exchange. When a partner says "we support secure FTP", always ask which one they mean, because the setup effort and the failure modes are completely different.

3. How an SFTP integration works

It helps to see the shape of a session end to end. An SFTP transfer is a sequence of well-defined phases: the client opens a TCP connection to port 22, the two sides negotiate SSH encryption and verify each other's identity, the client authenticates (usually with an SSH key), the SFTP subsystem starts, and then file operations flow inside that single encrypted channel until the client closes the session. The diagram below traces that flow, with the SSH key authentication and the encrypted channel that everything rides inside made explicit.

In an integration context, the client side is almost never a person at a keyboard. It is a scheduled job, a middleware connector, or an application module that runs unattended. That unattended nature is exactly why authentication and error handling matter so much: there is no human present to type a password, notice a warning about a changed host key, or retry a failed transfer. Everything has to be configured correctly in advance and then left to run on its own, often at three in the morning, often moving files that the business depends on being there when the working day starts. SFTP is a specific, secure instance of the broader file-based integration pattern, so if you want the pattern in general, including layout conventions and control files, see the file-based integration pillar, and for the payload format that most often rides inside these transfers, the CSV integration guide.

4. Authentication: passwords versus SSH keys

SFTP inherits its authentication options from SSH, and there are two that matter for integration: password authentication and public key authentication. Understanding the difference is the difference between a secure automated pipeline and an accident waiting to happen.

Password authentication is the obvious one. The client presents a username and a password, the server checks them, and if they match the session proceeds. This is fine for a human logging in interactively, but it is a poor fit for automation. To run an unattended job you have to store the password somewhere the job can read it, which means a plaintext or lightly obscured secret sitting in a config file, a scheduler definition or a script. It is guessable, it is subject to brute-force attempts against the exposed port, and rotating it means touching every job that uses it. Password authentication for machine-to-machine SFTP is a habit worth breaking.

Public key authentication is the right answer for integration, and it is how the diagram above works. The client generates a key pair: a private key that never leaves the client, and a public key that is safe to share. The public key is placed on the server in the target account's list of authorized keys. When the client connects, it proves it holds the matching private key through a cryptographic challenge, without ever transmitting the key itself. No shared secret crosses the network, there is nothing to brute-force on the exposed port, and you can authorise or revoke a specific client by adding or removing one public key on the server. It is both more secure and more operationally clean, which is a rare combination.

Practitioner's rule: use key-based authentication for every automated SFTP integration, and protect the private key like any other credential. Store it with tight filesystem permissions, keep it out of source control, and where the platform supports it, hold it in a secrets manager rather than a flat file. The public key on the server is not sensitive; the private key on the client is the crown jewel. Get that split right and most SFTP security problems disappear before they start.

A related decision is whether to protect the private key itself with a passphrase. For interactive use, a passphrase is a sensible extra layer. For fully unattended automation it creates a chicken-and-egg problem, because something has to supply the passphrase when no human is present, which usually pushes the secret back into a config file and defeats the point. The pragmatic pattern is a passphrase-less key with very tight filesystem permissions and, ideally, storage in a proper secrets vault, combined with server-side controls that lock the account down to nothing but file transfer. Which brings us to hardening.

5. Automating SFTP transfers (scheduling and watchers)

The whole point of SFTP in integration is that it runs without anyone watching. There are two dominant automation styles, and mature estates usually use both depending on the direction of the flow.

Scheduled transfers are the classic pattern. A job runs on a fixed timetable, a cron entry on Linux, a scheduled task on Windows, or a trigger inside an integration platform, connects to the remote server, and either pushes files up or pulls files down. This suits batch rhythms perfectly: pull last night's bank statement at 6am, push the payroll file every second Friday, collect the daily EDI drop at the top of each hour. Scheduling is simple, predictable and easy to reason about, and for the majority of file-based integrations it is all you need. The main design questions are how often to poll and what to do when the expected file is not there yet.

File watchers are the event-driven alternative. Instead of waking on a clock, a watcher monitors a directory, whether a local outbox or a remote inbox, and reacts the moment a new file appears. This gives you lower latency, files move as soon as they land rather than waiting for the next scheduled window, and it avoids the wasted polling cycles of a schedule that runs when there is nothing to do. Most enterprise integration tools and managed file transfer platforms provide directory-watching triggers for exactly this. The trade-off is a little more infrastructure and the need to handle the case where a file is still being written when the watcher fires, which is the classic partial-file trap I return to in the pitfalls section.

Whichever style you use, robust automation needs more than "connect and copy". It needs retry logic with sensible backoff for transient network failures, because remote servers go down and links drop. It needs alerting when a transfer fails or, just as importantly, when an expected file simply does not arrive, since a silent no-show is often more damaging than a loud error. It needs idempotency, so that re-running a failed job does not double-process or double-send. And it needs a clean archive-and-move discipline so that a processed file is moved out of the way and never picked up twice. These operational concerns are where amateur SFTP scripts and production-grade integrations part company, and they are worth building properly from the first day rather than bolting on after the first incident.

6. Security best practices

SFTP is secure by design, but a secure protocol configured carelessly is still a liability. The measures that matter most in practice:

• Use key-based authentication, not passwords. As covered above, this removes the shared secret from the network and from your config files, and lets you grant and revoke access one key at a time.
• Verify and pin host keys. The first time a client connects it learns the server's host key. Pin that known-good key in the client's configuration so that a later mismatch, which could signal a man-in-the-middle attack, causes the connection to fail loudly rather than silently trusting a new key.
• Restrict the account. An SFTP integration account should be able to do exactly one thing: transfer files into and out of its designated directories. Lock it into a chroot jail or its equivalent, deny it an interactive shell, and constrain it to the SFTP subsystem only. A file-transfer account that can also open a shell is a privilege-escalation gift.
• Apply least privilege on directories. Give each partner or flow its own directory with the minimum read or write permissions it needs. An inbound-only partner should not be able to read what other partners dropped.
• Keep the SSH server current. SFTP rides on SSH, so the security of your transfers depends on the SSH server being patched and configured to use strong, modern ciphers and key-exchange algorithms, with weak legacy ones disabled.
• Log and monitor transfers. Capture who connected, when, and what moved. An audit trail of file transfers is both a security control and an operational lifesaver when a downstream process complains that yesterday's file was wrong.

None of these are exotic, and all of them are cheap to apply at setup time and painful to retrofit after an incident. Because file-based flows so often carry exactly the sensitive data that attackers want, payroll, banking, personal records, the transfer layer deserves the same seriousness as any API boundary. For the wider treatment of securing integrations across every pattern, not just files, see the integration security best practices guide.

7. Common pitfalls (host keys, permissions, partial files)

The failures I have watched play out in real SFTP integrations are remarkably repetitive, and almost all of them are avoidable once you know the pattern.

Host key surprises. An unattended client that has not pinned the server's host key will either prompt on first connect, which a scheduled job cannot answer, or blindly accept whatever key it is offered, which defeats the security. Worse, when a server is rebuilt or migrated and its host key legitimately changes, a strict client will suddenly refuse to connect, and the transfer silently stops. The fix is to manage host keys deliberately: pin the known-good key, and treat any change as a controlled event with a documented update, not as something the client discovers at 3am.

Permission and ownership mistakes. SSH is famously strict about file permissions. A private key or an authorized-keys file that is readable by other users is often rejected outright, and integration jobs fail with cryptic authentication errors that have nothing to do with the key being wrong. On the server side, over-generous directory permissions leak data between partners, and under-generous ones stop a job writing where it should. Getting the permission model right, and understanding that SSH will refuse loose permissions on purpose, prevents a whole category of "it works from my laptop but not from the server" mysteries.

The partial-file trap, and why it deserves its own callout: the most damaging SFTP bug is also the most subtle. If a receiver picks up a file while the sender is still writing it, it processes half a file and corrupts downstream data, and often nobody notices until reconciliation fails days later. The standard defence is atomic delivery: the sender uploads to a temporary name or a staging directory, and only renames the file to its final, watched name once the upload is completely finished. A rename is atomic, so the watcher only ever sees a whole file. Alternatively, a small companion "trigger" or "done" marker file is written last, and the receiver waits for the marker before touching the payload. Build one of these in from day one; retrofitting it after a corruption incident is far more painful.

Timezone, encoding and line-ending drift. When two organisations exchange files, small assumptions diverge. A timestamp in the filename is read in the wrong timezone, a file encoded in one character set is parsed as another, or Windows and Unix line endings clash in a text payload. These are not SFTP problems as such, SFTP moved the bytes faithfully, but they surface at the file-transfer boundary and get blamed on it. Agreeing the filename convention, the encoding and the format explicitly with the partner, and validating on receipt, heads them off.

8. Where SFTP fits

With all of that in mind, when should you reach for SFTP, and when should you reach for something else? SFTP earns its place in a specific and enduring set of situations. It shines for cross-organisation batch exchange, where two parties want a secure, auditable, loosely-coupled way to move data without exposing live systems to each other. It shines for high-volume bulk data, nightly extracts, full-table dumps, large report files, where a batch file is more natural and more efficient than thousands of API calls. And it shines when the counterparty is a bank, a government body, a payroll bureau or a legacy platform that simply offers nothing but an SFTP endpoint, which in enterprise reality is often the deciding factor.

SFTP is the wrong tool when you need real-time or near-real-time data, where an API or an event stream fits far better, or when the interaction is genuinely transactional and needs an immediate response rather than a fire-and-collect-later rhythm. The honest framing is that file transfer and APIs are complementary, not competing: APIs for interactive, real-time, request-and-response integration, and SFTP for scheduled, bulk, cross-boundary batch movement. A mature integration estate uses both without embarrassment. Even platforms that lead with rich APIs still routinely exchange files this way behind the scenes, which is why the SFTP endpoint keeps appearing alongside the modern connectors. For how these patterns coexist inside one product's integration surface, the Business Central APIs and integrations guide is a good concrete example, and the broader picture lives in the enterprise system integration pillar.

9. References

A note on sources, because SFTP has an unusually messy standards history that is worth stating accurately rather than papering over.

• The SSH File Transfer Protocol (IETF SECSH drafts). SFTP was specified by the IETF Secure Shell (SECSH) working group as a series of internet drafts titled "SSH File Transfer Protocol". These drafts progressed through several versions but were never published as a finalised, ratified RFC. When people cite "the SFTP RFC", they are usually referring to one of these drafts. It is more accurate to describe SFTP as defined by the SECSH working group drafts than to attribute it to a single completed standard.
• The SSH protocol itself. Because SFTP runs as a subsystem of SSH, its security ultimately rests on the SSH transport, authentication and connection protocols, which are the ratified standards SFTP depends on. The encryption, host-key verification and public-key authentication described in this guide all belong to that SSH layer.
• OpenSSH. In practice, the de facto reference for SFTP behaviour is OpenSSH, the widely deployed open-source implementation that the vast majority of servers and clients speak. Where the drafts leave room for interpretation, "what OpenSSH does" is the working answer that real-world interoperability is built on.

I have deliberately not cited specific deep URLs or draft numbers here, because those move and get superseded; the reliable path is to search the IETF datatracker for the SECSH "SSH File Transfer Protocol" drafts and to consult the OpenSSH project documentation for implementation detail.

Final thoughts

SFTP is not glamorous, and that is precisely its strength. It has quietly moved the world's most sensitive routine data for two decades because it does one job extremely well: secure, reliable, auditable, cross-boundary file transfer over a single encrypted channel. The two ideas to carry away are simple. First, SFTP is the SSH File Transfer Protocol, a distinct protocol running over SSH, not a variant of FTP and not the same thing as FTPS, and confusing them will cost you time and firewall arguments. Second, the protocol's security is only as good as the way you operate it: use key-based authentication, pin host keys, restrict the account, and defend against partial files with atomic delivery.

Get those fundamentals right and SFTP will run for years without drama, which is exactly what you want from the layer that moves your payroll and your bank files while everyone sleeps. If you are designing a file-based integration, choosing between SFTP and FTPS with a partner, or trying to make an existing set of fragile transfer scripts production-grade, that is the kind of work I do, and the sequence, secure the transport, harden the account, make delivery atomic, then automate, is the same every time.

Related reading: Enterprise system integration explained (pillar), File-based integration, CSV integration, Integration security best practices, Business Central APIs and integrations.