LogoLogo
  • Welcome
  • Landing in Transparent Edge
  • Sign up process
  • Getting Started
    • Basics concepts
      • Glosary
        • API
        • Brotli Compression
        • Cache-Control
        • Cache key
        • Caching
        • CNAME
        • Cloud Computing
        • Cloud Computing Architecture
        • Cloud Services
        • DASH
        • Data Center
        • Edge Server
        • ETag
        • GSLB
        • HLS (HTTP Live Streaming)
        • HTTP/2
        • Infrastructure as a Service (IaaS)
        • Internet Exchange Point
        • Last-Modified
        • Load Balancing
        • MultiCDN
        • NoSQL (not only SQL)
        • Origin
        • Origin Shield
        • OTT (Over The Top)
        • Platform as a Service (PaaS)
        • PoP (Point of Presence)
        • Private CDN
        • Private Cloud
        • Public Cloud
        • Purge
        • Query String
        • Reverse Proxy
        • RTT (Round-trip Time)
        • SaaS (Software as a Service)
        • SDS (Software Defined Storage)
        • Smooth Streaming
        • Status Code
        • TCP (Transmission Control Protocol)
        • TLS Acceleration
        • TLS (Transport Layer Security)
        • TTFB (Time-to-first-byte)
        • TTL (Time-to-live)
        • Virtual Machine
        • VPS (Virtual Private Server)
        • Web Services
      • Let's start at the beginning
      • Things to consider
      • Houston, we have a problem
      • HTTP, How does it work?
      • Invalidating methods
      • DNS Pointing
      • Log formats
      • Predefined headers
      • Default headers
        • geo_country_code
        • X-Device
        • Vary
        • Cache headers
        • Age
        • TP-Cache
        • True-Client-IP and X-Forwarded-For
      • Forcing No-Cache
      • Architecture
        • Transparent Edge’s IP addresses
        • Locations and PoP
        • Cache layers
      • Cache effectiveness
      • SSL
      • HTTP 5xx Error Codes
      • Features
        • Protection against origin failures
        • Rate Limit
        • Geolocation and geoblocking
        • Prefechting
        • Refetching
        • Fast purging
        • HTTP Redirects
        • Caching static vs. dynamic objects
        • Rewriting of headers
        • Device detection
    • Dashboard
      • Historic
      • Analytics
      • Invalidating content
      • Content invalidation by tags
      • Prefetching Cache
      • Log shipping
      • Provisioning
        • Initial configuration
        • Backends
        • Sites
        • Configuration deployments
        • Network ACLs
        • TLS/SSL Certificates
      • User management
  • Configuration
    • VCL Reference
      • Default Functions
      • VCL Objects
      • Callable Functions
      • Security restrictions
      • Varnish book
    • Network Access Control List
      • Initial configuration
      • Auto generated lists
      • Manage lists via API
    • i3
      • Quality adjustment
      • Cache timing allocation for transformed images
      • Conversion to grayscale
      • Conversion to WebP
      • Blurring
      • Inclusion of graphics in the footer (strip)
      • Automatic resizing
      • Definition of the maximum size (content-length)
    • Transcoding
      • Relaunch or requeue jobs
      • Create a transcode job
      • Get job information
      • Dashboard usage
    • OpenAPI de TransparentCDN
  • Security
    • HTTPS
    • Blocking User-Agent
    • Blocking by IP Address
    • Blocking Requests Geographically
    • Avoiding Hotlinking
    • Bot Mitigation
    • WAF
      • Configuration
      • CAPTCHA
      • Content protected by token
      • Rate limit
    • Anomaly Detection
      • Detection Types
      • Automatic Reactions
      • Detection History
    • Under attack mode
    • Global Whitelists
  • Integrations
    • Wordpress plugin
    • Google Cloud Platform
    • Amazon Web Services
  • GUIDES AND TUTORIALS
    • How to do things
    • Edge Computing
      • ESI Tags
    • Acting on the Query String
    • Working with cookies
    • Making decisions based on HTTP headers
    • Web Application Gateway
    • Configure your servers to send cache headers
    • Caching a version per device
    • True-Client-IP in the origin
    • A/B Testing
    • Routing traffic to different backends
    • JSON Web Tokens
    • Debug codes
    • Streaming logs
    • API
      • Authentication
      • Invalidation
Powered by GitBook
On this page
  • Format and frequency
  • Log shipping to a FTP / sFTP server
  • Log shipping to an Amazon S3 bucket compatible
  • Sorting out compressed log files by date and time
  • Log sending by streaming

Was this helpful?

Export as PDF
  1. Getting Started
  2. Dashboard

Log shipping

PreviousPrefetching CacheNextProvisioning

Last updated 9 months ago

Was this helpful?

The log sending feature is included in any service package. It just needs to be activated and set up for it to start.

Format and frequency

In batch mode, logs are extracted, compressed and sent every hour. On average, there is one compressed log file for each edge node belonging to our delivery platform, provided requests have been processed on these and related with any of the sites involved.

Every compressed log file has the following name mask::

<YYYYMMDDHH>-<CID>-<COUNTRY CODE>-<HASH>.log.gz

Thereby:

  1. The first field corresponds to the send date (YYYYMMDDHH).

  2. The second is the unique client /company ID.

  3. The next one represents the country code associated to the geographical location of the edge node meeting those requests.

  4. Last but not least, a hash code is included which identifies the edge node itself. This field is used internally for traceability and debugging purposes.

For instance: 2023022010-4-spain-c7658c0dd514f6523f7a98ddd0dbb448.log.gz

Log shipping to a FTP / sFTP server

This choice allows the log to be sent to an arbitrary FTP / sFTP server.

To set it up properly, simply specify the correct protocol and fill in the gaps with the remaining parameters required. Once this choice is activated and its parameters validated, sending will begin within the next hour.

Log shipping to an Amazon S3 bucket compatible

This choice allows the log sending to a compatible Amazon S3 bucket.

In order to properly set it up, an access and secret key credential pair must be provided that must also be associated with a policy allowing, at least, file uploading into the bucket that is to be used.

Bucket field allows two name masks:

  • <ENDPOINT URL>/<BUCKET NAME>

  • <BUCKET NAME> (just in case of Amazon S3 bucket)

<ENDPOINT URL> corresponds to the URL hosting the S3 service. In the scenario where the bucket is hosted in Amazon S3, it is possible to use just the <BUCKET NAME> as the sole parameter.

Anywise, if it must also be included, for instance, to point to a specific geographical region, the bucket name must be removed from the domain itself: https://tcdn-testing.s3.us-east-1.amazonaws.com/tcdn-testing would become https://s3.us-east-1.amazonaws.com/tcdn-testing. If this is not done, the bucket name will be included at the beginning of the remote path.

Sorting out compressed log files by date and time

When log sending is being configured, path field, which represents the remote path where compressed log files will be stored, allows for a set of special date--related masks:

  • %Y, year

  • %m, month

  • %d, day

  • %H, hour

For instance, if the value of path field is /tcdn-logs/%Y/%m/%d and today is February 20, 2023, when the process is executed, the compressed log files will be remotely stored in the path /tcdn-logs/2023/02/20.

Log sending by streaming

Furthermore, there is another way of log sending to the client, in this case, in real time. Please check out to see more details on how to configure this feature.

Remember that every single task you can do from our can also be accomplished from our .

this link
dashboard
API