The Lua manual in section 6.4.1 on Lua Patterns states

A character class is used to represent a set of characters. The
following combinations are allowed in describing a character class:

• 
  x: (where x is not one of the magic characters ^$()%.*+-?) represents the character x itself.


• 
  .: (a dot) represents all characters.


• 
  %a: represents all letters.


• 
  %c: represents all control characters.


• 
  %d: represents all digits.


• 
  %g: represents all printable characters except space.


• 
  %l: represents all lowercase letters.


• 
  %p: represents all punctuation characters.


• 
  %s: represents all space characters.


• 
  %u: represents all uppercase letters.


• 
  %w: represents all alphanumeric characters.


• 
  %x: represents all hexadecimal digits.


• 
  %x: (where x is any non-alphanumeric character) represents the character x. This is the standard way to escape the magic characters.
  Any non-alphanumeric character (including all punctuation characters,
  even the non-magical) can be preceded by a % when used to represent
  itself in a pattern.


• 
  [set]: represents the class which is the union of all characters in set. A range of characters can be specified by separating the end
  characters of the range, in ascending order, with a -. All classes
  %x described above can also be used as components in set. All other
  characters in set represent themselves. For example, [%w_] (or
  [_%w]) represents all alphanumeric characters plus the underscore,
  [0-7] represents the octal digits, and [0-7%l%-] represents the
  octal digits plus the lowercase letters plus the - character.

You can put a closing square bracket in a set by positioning it as the
first character in the set. You can put a hyphen in a set by
positioning it as the first or the last character in the set. (You can
also use an escape for both cases.)

The interaction between ranges and classes is not defined. Therefore, patterns like [%a-z] or [a-%%] have no meaning.

[^set]: represents the complement of set, where set is interpreted
as above.

For all classes represented by single letters (%a, %c, etc.), the
corresponding uppercase letter represents the complement of the class.
For instance, %S represents all non-space characters.

The definitions of letter, space, and other character groups depend on
the current locale. In particular, the class [a-z] may not be
equivalent to %l.
_{(Highlighting and some formatting added by me)}

So, since the "interaction between ranges and classes is not defined.", how do you create a character class set that starts and/or ends with a (magic) character that needs to be escaped?

For example,

[%%-c]

does not define a character class that ranges from % to c and includes all characters in-between but a set that consists only of the three characters %, -, and c.

The interaction between ranges and classes is not defined.

Obviously, this is not a hard and fast rule (of regex character sets in general) but a Lua implementation decision. While using shorthand characters in character sets/ranges work in some (most) regex flavors, it does not in all (like in Python's re module, demo).

However, the second example is misleading:

Therefore, patterns like [%a-z] or [a-%%] have no meaning.

While the first example is fine since %a is a shorthand class (that represents all letters) in a set, [%a-z] is undefined and will return nil if matched against a string.

Escaped range characters in a [set]

In the second example, [a-%%], %% simply defines an escaped % sign and not a shorthand character class. The superficial problem is, the range is defined upsidedown, from high to low (in reference to the US ASCII value of the characters a 61 and % 37), e.g like an erroneous Lua pattern like [f-a]. If the set is defined in reverse order it seems to work: [%%-a] but all it does is matching the three individual characters instead of the range of characters between % and a; credit cyclaminist).

This could be considered a bug and, indeed, means it is not possible to create a range of characters in a [set] if one of the defining range characters need to be escaped.

Possible Solution

Start the character range from the next character that does not need to be escaped - and then add the remaining escaped characters individually, e.g.

[%%&-a]

Sample:

for w in string.gmatch("%&*()-0Aa", "[%%&-a]") do
 print(w)
end

This is the answer I have found. Still, maybe somebody else has something better.