CloudBuddy Analytics

    What is CloudBuddy Analytics?

  • CloudBuddy Analytics is an Open Source web based tool to generate exhaustive statistical reports about your S3 Bucket access. It has an intuitive interface for a rich user experience. CloudBuddy Analytics takes care of enabling logging, fetching logs and generating reports.
  • CloudBuddy Analytics can be configured for Multiple S3 Accounts, and also could be made available on the network as a Multi User tool.
  • Why CloudBuddy Analytics?

  • CloudBuddy Analytics can be installed locally, without the need for sharing you access credentials. Since its an Open Source tool, it is easily extensible and is open to see its mechanics.
  • CloudBuddy Analytics uses a simple, yet effective caching algorithm to give you faster reports. And the use of AWStats engine, means your reports are exhaustive and accurate.
  • Apart from that, CloudBuddy Analytics has an extremely easy Windows setup mechanism, and is packaged with a compact, yet extremely powerful, Abyss WebServer.
  • CloudBuddy Analytics Features?

  • CloudBuddy Analytics comes with the following features.
    • Configure Multiple AWS Accounts
    • Caching for faster reports
    • Bucket log enabling capability
    • Monthly wise reports
    • Bandwidth usage
    • Number of visits, number of unique visitors, visit durations and last visits
    • Days of week and rush hours (pages, hits, KB for each hour and day of week)
    • Domains/countries of hosts visitors (pages, hits, KB, 269 domains/countries detected, GeoIP detection)
    • Most viewed files/pages
    • Most accessed file types
    • OS used (pages, hits, KB for each OS, 35 OS detected)
    • Browsers used
    • Search engines, keyphrases and keywords used to find your site (The 115 most famous search engines are detected like yahoo, google, altavista, etc.)
    • HTTP errors (Page Not Found with last referrer, ...)
    • Hostname resolution


  • This is the breakdown of the CloudBuddy Analytics into undestandable steps to showcase whats going on in the background. The following is the summary of what user will be put through to see his bucket statistics:-
    • User-Account Creation
    • Login
    • Buckets Analytics Subscription
    • Add New Account
    • Remove Account
    • Buckets Analytics Unsubscribe
    • View Analytics
    • Logout
  • User-Account Creation.

  • To create an account the user is prompted to provide with following details.
    • Username
    • Password
    • Confirm password
    • Account Name
    • AWS Access Key
    • AWS Secret Key
  • The user-account creation has two levels of validation.
    • The AWS account validation with the AWS.
    • The user validation with the Analytics DB.
  • On passing the above criteria, user-account creation will make an entry into two tables cba_users and cba_accounts which will have the following schema.
  • cba_users
  • varchar(25) unique

  • cba_accounts
  • varchar(25)
    varchar(50) unique

  • Note: username+accountname will need to be unique.
  • The default needs a mention here that for now the account provided during the user creation will automatically become the default account of that user.
  • User Login

  • The User Login can happen in two ways.
    • On User-Account creation.
    • On user login with existing User-Account.
  • One small step happens in the latter case. The user is validated with the Analytics database table cba_users and default account is picked up from the cba_accounts.
  • Then the procedure is the same for both.
    • The accesskey and secretkey as picked up for the default account from cba_accounts
    • List of buckets for that accesskey and secretkey are retrieved to become the Bucket list.
  • Bucket Analytics Subscription

  • On selecting a bucket from the bucket listing the cba_buckets table, which has the following schema, is checked against the bucketname.
  • cba_buckets
  • varchar(50)
    varchar(256) unique
    varchar(256) unique

  • If there is an entry in the table for that bucket...
  • ...the "View Analytics" along with "Unsubscribe".
  • else...
  • ...the "Subscribe" link is displayed.
  • Clicking on the Subscribe link will make cgi call to our Customized AWStats. Which in short will...
    • Add the accesskey and bucketname to the cba_buckets
    • If logging already enabled...
      ...retrieve the logs.
      ...update the targetbucket, prefix and lastlogtime.
      ...display statistics.
      ...enable the logging.
      ...update the targetbucket and prefix.
      ...tell the user that it will take while. "Pls revisit"
  • Add New Account.

  • The add new account will will add new account for the user who asked for it.
    • Prompt the user for the accountname and AWS credentials.
    • Validate accountname for duplication, and AWS credentials.
    • Invalid: Go to step 1.
    • Valid: Add username, accountname, accesskey and secretkey to the cba_accounts table.
    • Retrieve the bucket list for the new account.
  • Remove Account.

  • The removal of account is a little restrictive at this moment. As the use is not allow to create himself without giving a valid AWS account, the user will not be allowed to remove all his account. i.e., the user will always have at least one account.
  • The removal of an user is fairly simple apart from the above constraint. It is just deleting the account entry from the cba_accounts.
  • Buckets Analytics Unsubscribe

  • On selecting a bucket that has an entry in cba_buckets the user has two possible actions and one of them is Unsubscribe to analytics. When the user selects to unsubscribe, it needs to be confirmed upon which the following is carried out by the Customized AWStats.
    • Deleted all the cached log data for the corresponding bucket.
    • Delete the archived log data from the archive bucket.
    • Remove bucket entry from the cba_buckets
  • View Analytics

  • The other action that user can carry out is the "View Analytics". This completely handled by the Customized AWStats. In short this is what happens:
    • Checks for awstatsdb files locally.
    • If NOT above, the compressed log files are downloaded from the S3.
    • lastlogtime is checked and logs are downloaded from the target bucket.
    • Logs are merged into one single .log file.
    • log is parsed by awstats.
    • Awstats db files, month-wise, are generated.
    • lastlogtime is updated.
    • The logs are compressed and uploaded to S3.
    • Analytics are displayed.
  • Logout

  • Logout takes you back to the login page.