Animating sprites in DragonRuby is fairly simple, but it does require putting a couple ideas together.

First, it’s best to have a single image with all frames of the animation together, equally spaced apart. I prefer the frames are arranged horizontally from left-to-right, so that is what we will use here.

Here is an example, borrowed from here:

The first frame can be displayed like this:

def tick args
  height = 195
  width = 192

  args.outputs.sprites << {
    x: args.grid.center_x - (width / 2),
    y: args.grid.center_y - (height / 2),
    h: height,
    w: width,
    source_x: 0,
    source_y: 0,
    source_w: width,
    source_h: height,
    path: 'sprites/walking.png',
  }
end

source_x and source_y set the bottom left corner of a “tile” or basically a slice of the image. (To use the top left instead, set tile_x and tile_y). source_w and source_h set the width and height of the tile. The sprite can be scaled when displayed with w and h.

If the frames are laid out horizontally, then all one needs to do is update the source_x value (typically by the width of the tile) in order to change the frame.

Here is an illustration for a few frames:

We could accomplish this by using the multiplying the width of the tile by the current tick (modulo the number of frames, so it loops):

def tick args
  height = 195
  width = 192
  num_frames = 8

  source_x = width * (args.tick_count % num_frames)

  args.outputs.sprites << {
    x: args.grid.center_x - (width / 2),
    y: args.grid.center_y - (height / 2),
    h: height,
    w: width,
    source_x: source_x,
    source_y: 0,
    source_w: width,
    source_h: height,
    path: 'sprites/walking.png',
  }
end

This works… but it’s a bit fast for a walk!

This is where DragonRuby helps out. The frame_index method will do the calculation of the current frame for us.

frame_index accepts these arguments:

• count: total number of frames in the animation
• hold_for: how many ticks to wait between frames
• repeat: whether or not to loop

frame_index can be called on any integer, but typically uses the tick number on which the animation started. Below, the code sets this to 0 (the first tick). This could instead be when an event happens, based on input, or anything else.

Multiplying the width of the tile by the frame index results in the source_x value for the current frame of the animation:

def tick args
  height = 195
  width = 192
  num_frames = 8
  start_tick = 0
  delay = 4

  source_x = width * start_tick.frame_index(count: num_frames, hold_for: delay, repeat: true)

  args.outputs.sprites << {
    x: args.grid.center_x - (width / 2),
    y: args.grid.center_y - (height / 2),
    h: height,
    w: width,
    source_x: source_x,
    source_y: 0,
    source_w: width,
    source_h: height,
    path: 'sprites/walking.png',
  }
end

And that’s it!

But With Ruby Classes

Once a game starts to get moderately complex, I like to arrange behavior into classes. It’s also convenient to use attr_gtk to avoid passing args around and to save on some typing (e.g. args.outputs becomes just outputs).

class MyGame
  attr_gtk

  def initialize(args)
    @my_sprite = MySprite.new(args.grid.center_x, args.grid.center_y)
    args.outputs.static_sprites << @my_sprite
  end

  def tick
    if inputs.mouse.click
      if @my_sprite.running?
        @my_sprite.stop
      else
        @my_sprite.start(args.state.tick_count)
      end
    end

    @my_sprite.update
  end
end

class MySprite
  attr_sprite

  def initialize x, y
    @x = x
    @y = y

    @w = 192
    @h = 195
    @source_x = 0
    @source_y = 0
    @source_w = @w
    @source_h = @h
    @path = 'sprites/walking.png'

    @running = false
  end

  # Set @running to the current tick number
  # this is so the frame_index can use that as the
  # start of the animation timing.
  def start(tick_count)
    @running = tick_count
  end

  def stop
    @running = false
  end

  def running?
    @running
  end

  # Update source_x based on frame_index
  # if currently running
  def update
    if @running
      @source_x = @source_w * @running.frame_index(count: 8, hold_for: 4, repeat: true)
    end
  end
end

def tick args
  $my_game ||= MyGame.new(args)
  $my_game.args = args
  $my_game.tick
end

This example essentially follows my Object-Oriented Starter approach and moves the logic into a game class and a sprite class.

When the mouse is clicked, the sprite starts moving (using the current tick_count as the starting tick). When the mouse is clicked again, the sprite stops.

Source vs. Tile

To use just a piece of an image (for animations or otherwise), there are two options: source_(x|y|h|w) or tile_(x|y|h|w).

These options are nearly identical, except source_y is bottom left and tile_y is top left.

The source_ options were added in DragonRuby 1.6 and are more consistent with the rest of DragonRuby where the origin is the bottom left. On the other hand, the tile_ options align easier with image editors.

Either option works, depending on what is important to you.

Go!

Now that’s really it! Get moving!

In DragonRuby, one of the drawing primitives is a “solid” - a rectangle, actually.

Rectangles are defined by the origin point of the bottom-right corner (x, y) and a size in height/width (h, w).

For example, this code paints a black rectangle in roughly the middle of the screen:

def tick(args)
  args.outputs.solids << {
    x: 490,
    y: 310,
    w: 300,
    h: 100,
  }
end

Rectangles are simple and (presumably) fast. But what if we want to put the rectangle at an angle? Or spin it around?

For sprites, there is an angle attribute. Will that work for rectangles? Unfortunately, no.

We could make a rectangle image and use it in a sprite, but that seems wasteful. I don’t know if resizing a sprite is very resource-intensive, but I’m sure it takes more cycles than changing the size of a simple rectangle.

Fortunately, there is a middle way.

From the 2.26 release notes (and yes, this is the only place I could find this functionality formally documented, though it is used in examples):

** [API] Pre-defined ~:pixel~ render target now available. Before ~(boot|tick)~ are invoked, a white solid with a size of 1280x1280 is added as a render target. You can use this predefined render target to create solids and get ~args.outputs.sprites~ related capabilities.

In other words, it is possible to create an equivalent rectangle to the above like this:

def tick(args)
  args.outputs.sprites << {
    x: 490,
    y: 310,
    w: 300,
    h: 100,
    r: 0,
    g: 0,
    b: 0,
    path: :pixel,
  }
end

The key piece is path: :pixel and creating a sprite instead of a solid.

As is mentioned above, the :pixel render target is white, so to get exactly the same results as before the color is set to black. If no rgb values were specified, it would be a white rectangle.

Now, can we rotate this rectangle?

Sure!

def tick(args)
  args.outputs.sprites << {
    x: 490,
    y: 310,
    w: 300,
    h: 100,
    r: 0,
    g: 0,
    b: 0,
    path: :pixel,
    angle: 45,
  }
end

(By the way, angles in DragonRuby are in degrees.)

Spinning Round and Round

Here is an example of spinning the rectangle around:

def tick(args)
    args.outputs.sprites << {
      x: 490,
      y: 310,
      w: 300,
      h: 100,
      r: 0,
      g: 0,
      b: 0,
      path: :pixel,
      angle: args.tick_count % 360,
    }
end

This is great and all, but what if you’d rather have it spin around like this?

With some sleuthing, you might find the angle_anchor_x and angle_anchor_y attributes. And you would be forgiven for thinking those refer to points on the coordinate grid. But they do not!

Instead, angle_anchor_x and angle_anchor_y are percentages relative to the sprite itself.

In other words, if both anchors are set to 0.5 (the default), the center of the rotation will be the middle of the sprite (half of the width and half of the height).

0,0 is the bottom-left corner of the sprite, as shown above:

def tick(args)
    args.outputs.sprites << {
      x: 490,
      y: 310,
      w: 300,
      h: 100,
      r: 0,
      g: 0,
      b: 0,
      path: :pixel,
      angle: args.tick_count % 360,
      angle_anchor_x: 0,
      angle_anchor_y: 0,
    }
end

Anchor values between 0 and 1 will be inside the sprite. But values greater than 1 or less than 0 will be outside the sprite.

Here is a little reference:

Happy rectangle rotating!

If you are using Cloudflare, it can be helpful to configure Cloudflare to push request logs to S3. Otherwise, the Cloudflare dashboard provides only a limited view into your data (72 hours at a time and sampled data instead of full logs).

Once the Cloudflare request logs are in S3, they can be queried using Athena. This blog post even provides a nice CREATE TABLE command to set up the table in Athena.

However, there is a problem. When performing a query in Athena, it might have to scan all of the logs in S3, even if you try to limit the query. This can be slow and costly, as Athena queries are charged per byte scanned.

The only way to really limit the amount of data scanned is to partition the data.

This post assumes you have already set up Cloudflare to push logs to an S3 bucket, configured a database in Athena to access it, and then realized those logs will grow forever, along with your query times.

(If you just want the “how to” without the exposition, jump down to “Setting Up Partitions for Cloudflare Logs”.)

Partitioning

Most commonly, you will want to look at logs from a specific time period, so it makes sense to partition the logs by date.

Most of the partitioning documentation suggests the files (or in this case, S3 objects) include a column=value key pair in the name. The column can then be used as a partition.

Unfortunately, Cloudflare does not allow customizing the format of the file names it produces.

Fortunately, the file names do include date/time information. The logs are grouped by date and time range:

s3://mah_s3_bucket/20210812/20210812T223000Z_20210812T224000Z_9af500e2.log.gz

So all we need to do is grab that date “folder” name and that’s our partition! Easy, right?

No, wrong. This is AWS. Nothing is easy.

Use a Recurring Job?

Several of the AWS documentation pages suggest using ALTER TABLE to ADD PARTITIONs.

Something like this:

ALTER TABLE cloudflare_logs ADD
  PARTITION (dt = '2021-08-12') LOCATION 's3://mah_log_bucket/20210812/';

But since the logs will grow every day, we’ll need to add new a new partition every 24 hours. It is not possible to “predefine” the partitions.

This requires setting up a recurring job… somewhere… to periodically define the new partitions. So now we have to pull in another AWS service to make S3 and Athena work nicely?! No thanks!

Partition Projection

At the bottom of the page about partitions, there is a paragraph about “partition projection” that sounds promising:

To avoid having to manage partitions, you can use partition projection. Partition projection is an option for highly partitioned tables whose structure is known in advance.

Yes, this is what we want! But how does it work?

Essentially like this:

We must define a column, its type, start and end values, and the interval between those values. Athena will then be able to extrapolate all the possible values.

Then we define a pattern to pull the value out of each S3 object name. This allows Athena to figure out which objects (logs) are associated with which partition value.

For example, the partition 20210812 will be associated with s3://mah_s3_bucket/20210812/20210812T223000Z_20210812T224000Z_9af500e2.log.gz

Once that’s all done, we can query based on the partition as if it were a column, like:

SELECT * FROM cloudflare_logs
WHERE log_date >= '20210812'
  AND log_date < '20210901';

Setting Up Partitions for Cloudflare Logs

Here are the steps that must be taken to set up the partitions:

1. Add the partition “column” when creating the table
2. Set several properties on the table to define the projection
3. Set the partition pattern to match against object names
4. Enable projection

(Actually, 2-4 are all the same: set (totally unvalidated) key-value properties on the table.)

Fortunately, all of this can be accomplished with one giant Athena command:

CREATE EXTERNAL TABLE `YOUR_TABLE_NAME`(
  `botscore` int,
  `botscoresrc` string,
  `cachecachestatus` string,
  `cacheresponsebytes` int,
  `cacheresponsestatus` int,
  `clientasn` int,
  `clientcountry` string,
  `clientdevicetype` string,
  `clientip` string,
  `clientipclass` string,
  `clientrequestbytes` int,
  `clientrequesthost` string,
  `clientrequestmethod` string,
  `clientrequestpath` string,
  `clientrequestprotocol` string,
  `clientrequestreferer` string,
  `clientrequesturi` string,
  `clientrequestuseragent` string,
  `clientsslcipher` string,
  `clientsslprotocol` string,
  `clientsrcport` int,
  `edgecolocode` string,
  `edgecoloid` int,
  `edgeendtimestamp` string,
  `edgepathingop` string,
  `edgepathingsrc` string,
  `edgepathingstatus` string,
  `edgeratelimitaction` string,
  `edgeratelimitid` int,
  `edgerequesthost` string,
  `edgeresponsebytes` int,
  `edgeresponsecontenttype` string,
  `edgeresponsestatus` int,
  `edgeserverip` string,
  `edgestarttimestamp` string,
  `firewallmatchesactions` array<string>,
  `firewallmatchesruleids` array<string>,
  `firewallmatchessources` array<string>,
  `originip` string,
  `originresponsestatus` int,
  `originresponsetime` int,
  `originsslprotocol` string,
  `rayid` string,
  `wafaction` string,
  `wafflags` string,
  `wafmatchedvar` string,
  `wafprofile` string,
  `wafruleid` string,
  `wafrulemessage` string,
  `workersubrequest` boolean,
  `zoneid` bigint)
PARTITIONED BY (
  `YOUR_COLUMN_NAME` string)
ROW FORMAT SERDE
  'org.openx.data.jsonserde.JsonSerDe'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat'
LOCATION
  's3://YOUR_BUCKET_NAME/'
TBLPROPERTIES (
  'projection.enabled'='TRUE',
  'projection.YOUR_COLUMN_NAME.format'='yyyyMMdd',
  'projection.YOUR_COLUMN_NAME.interval'='1',
  'projection.YOUR_COLUMN_NAME.interval.unit'='DAYS',
  'projection.YOUR_COLUMN_NAME.range'='YOUR_START_DATE,NOW',
  'projection.YOUR_COLUMN_NAME.type'='date', 
 
 'storage.location.template'='s3://YOUR_BUCKET_NAME/${YOUR_COLUMN_NAME}/'
) 

These are the pieces related to partitioning:

PARTITIONED BY (
  `YOUR_COLUMN_NAME` string)

This tells Athena to set up a column for the partition.

(The type of the partition column must be string, even though the projection type must be date. Does this make any sense? NO.)

Then the table properties.

Turn on projection:

'projection.enabled'='TRUE'

Set the column type to date:

'projection.YOUR_COLUMN_NAME.type'='date'

This is so Athena knows how to interpolate values.

Define the date format to match:

'projection.YOUR_COLUMN_NAME.format'='yyyyMMdd'

Set the interval for the values to one day:

'projection.YOUR_COLUMN_NAME.interval'='1',
'projection.YOUR_COLUMN_NAME.interval.unit'='DAYS'

Set the range for the values:

'projection.YOUR_COLUMN_NAME.range'='YOUR_START_DATE,NOW'

NOW is a special value so the end of the range will always be the current day.

This sets a template to extract the date string from the object name, using the date template defined above, and setting the value in the column name specified for the projection:

'storage.location.template'='s3://YOUR_BUCKET_NAME/${YOUR_COLUMN_NAME}/'

If you are unfamiliar with Athena, it’s good to know that deleting/creating tables is low impact. If the table is already created, it is not a big deal to delete it and start over.

Here are the important bits above that you will need to change:

• YOUR_TABLE_NAME is whatever you want to name the table. Something like cloudflare_logs would probably make sense.
• YOUR_COLUMN_NAME is whatever you want to name the projection “column”. Could be dt like in the AWS docs, or log_date or whatever you want.
• YOUR_BUCKET_NAME is the name of the S3 bucket.
• YOUR START_DATE is the date of the first log. Something like 20210101.

The table should now indicate it is partitioned:

And the partition should show up as a column:

Using Date Partitions

To test if partitions are working as expected, a quick query like this will work:

SELECT DISTINCT(YOUR_COLUMN_NAME)
FROM "YOUR_TABLE_NAME"
LIMIT 10;

The expected output is several dates for which there are logs.

Once that is confirmed, the partition column can be used like any other column. Since the values is a basic ISO date format, comparison operators can be safely used even though the column is really just a string.

For a single day:

SELECT * FROM YOUR_TABLE
WHERE YOUR_COLUMN_NAME = "20200101";

For an inclusive range:

SELECT * FROM YOUR_TABLE
WHERE YOUR_COLUMN_NAME >= "20200101"
  AND YOUR_COLUMN_NAME <= "20210101";

And now your queries can be faster and cheaper!