Files on S3 (Parquet, CSV)

Read Parquet and CSV files from an Amazon S3 bucket or an S3-compatible store (MinIO, Ceph, Cloudflare R2 and others). Each table you declare is a file pattern; all matching files are imported together into one Data Table. Fresco BI only lists and reads files; it never writes to the bucket.

Feature	Supported
Import	Yes
DirectQuery	No
SSL/TLS	Always on (HTTPS)
SSH tunnel	No
Cloud	Yes
On-premises (via Gateway)	No

Prerequisites

• A bucket, and the folder (prefix) that holds the files.
• An access key with read-only rights on that bucket and prefix.
• For a non-AWS endpoint: an organization admin adds its host name in Organization settings › Allowed S3 endpoints first (up to 20 hosts). AWS S3 endpoints are always allowed.

Step 1: Create a read-only access key

On AWS, create an IAM user (or role) with only this policy, then create an access key for it. Replace <bucket> and <prefix> (the prefix ends with /; use the policy without the prefix parts to allow the whole bucket):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::<bucket>",
      "Condition": { "StringLike": { "s3:prefix": "<prefix>*" } }
    },
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::<bucket>/<prefix>*"
    }
  ]
}

Do not grant s3:PutObject, s3:DeleteObject or s3:*.

On an S3-compatible store, create a user or key with read and list rights on the bucket only.

Temporary credentials work too: enter the session token with the key.

Step 2: Allow network access

Cloud. Fresco BI connects to the endpoint over HTTPS (port 443 unless you set another port). If the bucket or store limits client IPs (for example an aws:SourceIp condition in the bucket policy, or a firewall in front of MinIO), allow Fresco BI’s outbound IP addresses (the setup guide beside the data source form lists them).

The endpoint must resolve to a public address; private, loopback and metadata addresses are refused. There is no SSH tunnel and no On-premises (Gateway) option for this connector.

Step 3: Fill in the form

The web form for Files on S3 is not available yet. The source takes these settings:

1. Endpoint: a host name only, no https://, port or path; IP addresses are refused. Default s3.amazonaws.com. AWS S3 hosts (s3.amazonaws.com, s3.<region>.amazonaws.com) are always allowed; other hosts must be on the organization’s allowed list.
2. Port: optional; leave it empty for 443.
3. Region: for example eu-west-1, or auto (Cloudflare R2). Default us-east-1.
4. Bucket: the bucket name.
5. Prefix: optional folder, ending with /, no wildcards, for example exports/. Fresco BI cannot read outside s3://<bucket>/<prefix>.
6. URL style: vhost (default, bucket.endpoint) or path (endpoint/bucket, common for MinIO).
7. Access key ID, Secret access key and optional Session token. They are stored encrypted and never shown again.
8. Tables: 1 to 100, with unique names. Each table has a Table name, a File pattern relative to the prefix (for example sales/**/*.parquet) and a Format: parquet (default) or csv.
9. CSV options (CSV tables only): Delimiter , (default), ;, tab or |; First row is the header (default on); Quote " (default) or '; Skip rows 0 to 100.

Patterns use S3 key characters with *, ** and ?; braces, brackets and .. are refused. A Parquet table reads files ending in .parquet; a CSV table reads .csv, .csv.gz, .csv.zst and .tsv. Other files that match the pattern are ignored.

Limits

• At most 1,000 files per table and 20 GiB per table in total.
• At most 5 GiB per Parquet file and 2 GiB per CSV file.
• At most 500 columns per table.
• The file list is fixed when a load starts. If the matching files change during the load, the load stops instead of mixing old and new files; run it again.

Troubleshooting

• The server rejected the user name or password. S3 answered 401 or 403. Check the keys, and that the policy gives s3:ListBucket and s3:GetObject on this bucket and prefix (a missing permission also shows this message).
• The database or object was not found on the server. The bucket does not exist, or no file with the right extension matches the pattern. Check the bucket, prefix, pattern and format.
• The connection settings are not valid: check the marked fields. Often a wrong region (S3 answered with a redirect or 400), or a non-AWS endpoint that is not on the organization’s allowed list.
• A size or concurrent-connection limit was exceeded. Too many files, a file or table over the size limit, too many columns, or not enough memory. Narrow the pattern or split the table.
• The source returned a data type this connector does not support. A file is not valid Parquet or CSV. Check the format and the CSV options.
• The source table no longer matches the data table. The matching files changed during the load, or a column the Data Table reads is gone from the files. Run the load again, or check the files.
• The SSL connection could not be set up with the chosen mode. Check the SSL mode and the CA certificate. The endpoint’s certificate is not trusted by a public CA. A private CA cannot be added for this connector.
• The host is in a blocked range (private, loopback or metadata). A database inside a private network needs an On-premises connection through a Gateway. The endpoint resolves to a private address. Use a public endpoint.
• No connection could be opened to this host and port. Check the host, the port, and that the firewall allows Fresco BI’s IP. Check the endpoint, the port and any IP restriction on the store.