summaryrefslogtreecommitdiff
path: root/03-lexical-syntax.md
blob: 6230bea8b343489ce8732a913303498b16aed70a (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
Lexical Syntax
==============

Scala programs are written using the Unicode Basic Multilingual Plane
(_BMP_) character set; Unicode supplementary characters are not
presently supported.  This chapter defines the two modes of Scala's
lexical syntax, the Scala mode and the _XML_ mode. If not
otherwise mentioned, the following descriptions of Scala tokens refer
to Scala mode, and literal characters ‘c’ refer to the ASCII fragment
\\u0000-\\u007F

In Scala mode, _Unicode escapes_ are replaced by the corresponding
Unicode character with the given hexadecimal code.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
UnicodeEscape ::= \{\\}u{u} hexDigit hexDigit hexDigit hexDigit
hexDigit      ::= ‘0’ | … | ‘9’ | ‘A’ | … | ‘F’ | ‘a’ | … | ‘f’
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To construct tokens, characters are distinguished according to the following 
classes (Unicode general category given in parentheses):

1. Whitespace characters. `\u0020 | \u0009 | \u000D | \u000A`
1. Letters, which include lower case letters(Ll), upper case letters(Lu),
   titlecase letters(Lt), other letters(Lo), letter numerals(Nl) and the 
   two characters \\u0024 ‘\\$’ and \\u005F ‘_’, which both count as upper case 
   letters
1. Digits `‘0’ | … | ‘9’`
1. Parentheses `‘(’ | ‘)’ | ‘[’ | ‘]’ | ‘{’ | ‘}’ `
1. Delimiter characters ``‘`’ | ‘'’ | ‘"’ | ‘.’ | ‘;’ | ‘,’ ``
1. Operator characters. These consist of all printable ASCII characters
   \\u0020-\\u007F which are in none of the sets above, mathematical symbols(Sm) 
   and other symbols(So).

\pagebreak[1]

Identifiers
-----------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
op       ::=  opchar {opchar} 
varid    ::=  lower idrest
plainid  ::=  upper idrest
           |  varid
           |  op
id       ::=  plainid
           |  ‘`’ stringLit ‘`’
idrest   ::=  {letter | digit} [‘_’ op]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

There are three ways to form an identifier. First, an identifier can
start with a letter which can be followed by an arbitrary sequence of
letters and digits. This may be followed by underscore ‘_’
characters and another string composed of either letters and digits or
of operator characters.  Second, an identifier can start with an operator 
character followed by an arbitrary sequence of operator characters.
The preceding two forms are called _plain_ identifiers.  Finally,
an identifier may also be formed by an arbitrary string between
back-quotes (host systems may impose some restrictions on which
strings are legal for identifiers).  The identifier then is composed
of all characters excluding the backquotes themselves.
 
As usual, a longest match rule applies. For instance, the string

~~~~~~~~~~~~~~~~ {.scala}
big_bob++=`def`
~~~~~~~~~~~~~~~~

decomposes into the three identifiers `big_bob`, `++=`, and
`def`. The rules for pattern matching further distinguish between
_variable identifiers_, which start with a lower case letter, and
_constant identifiers_, which do not.

The ‘\$’ character is reserved
for compiler-synthesized identifiers.  User programs should not define
identifiers which contain ‘\$’ characters.

The following names are reserved words instead of being members of the
syntactic class `id` of lexical identifiers.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
abstract    case        catch       class       def
do          else        extends     false       final
finally     for         forSome     if          implicit
import      lazy        match       new         null
object      override    package     private     protected
return      sealed      super       this        throw       
trait       try         true        type        val         
var         while       with        yield
_    :    =    =>    <-    <:    <%     >:    #    @
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The Unicode operators \\u21D2 ‘$\Rightarrow$’ and \\u2190 ‘$\leftarrow$’, which have the ASCII 
equivalents ‘=>’ and ‘<-’, are also reserved.

(@) Here are examples of identifiers:

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        x         Object        maxIndex   p2p      empty_?
        +         `yield`       αρετη     _y       dot_product_*
        __system  _MAX_LEN_     
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

(@) Backquote-enclosed strings are a solution when one needs to
    access Java identifiers that are reserved words in Scala. For
    instance, the statement `Thread.yield()` is illegal, since
    `yield` is a reserved word in Scala. However, here's a
    work-around: `` Thread.`yield`() ``{.scala}


Newline Characters
------------------

~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
semi ::= ‘;’ |  nl {nl}
~~~~~~~~~~~~~~~~~~~~~~~~

Scala is a line-oriented language where statements may be terminated by
semi-colons or newlines. A newline in a Scala source text is treated
as the special token “nl” if the three following criteria are satisfied:

1. The token immediately preceding the newline can terminate a statement.
1. The token immediately following the newline can begin a statement.
1. The token appears in a region where newlines are enabled.

The tokens that can terminate a statement are: literals, identifiers
and the following delimiters and reserved words:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
this    null    true    false    return    type    <xml-start>    
_       )       ]       }
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The tokens that can begin a statement are all Scala tokens _except_
the following delimiters and reserved words:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
catch    else    extends    finally    forSome    match        
with    yield    ,    .    ;    :    =    =>    <-    <:    <%    
>:    #    [    )    ]    }
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A `case`{.scala} token can begin a statement only if followed by a
`class`{.scala} or `object`{.scala} token.

Newlines are enabled in:

1. all of a Scala source file, except for nested regions where newlines
   are disabled, and
1. the interval between matching `{` and `}` brace tokens,
   except for nested regions where newlines are disabled.

Newlines are disabled in:

1. the interval between matching `(` and `)` parenthesis tokens, except for
   nested regions where newlines are enabled, and
1. the interval between matching `[` and `]` bracket tokens, except for nested
   regions where newlines are enabled.
1. The interval between a `case`{.scala} token and its matching
   `=>`{.scala} token, except for nested regions where newlines are
   enabled.
1. Any regions analyzed in [XML mode](#xml-mode).

Note that the brace characters of `{...}` escapes in XML and
string literals are not tokens, 
and therefore do not enclose a region where newlines
are enabled.

Normally, only a single `nl` token is inserted between two
consecutive non-newline tokens which are on different lines, even if there are multiple lines
between the two tokens. However, if two tokens are separated by at
least one completely blank line (i.e a line which contains no
printable characters), then two `nl` tokens are inserted.

The Scala grammar (given in full [here](#scala-syntax-summary))
contains productions where optional `nl` tokens, but not
semicolons, are accepted. This has the effect that a newline in one of these
positions does not terminate an expression or statement. These positions can
be summarized as follows:

Multiple newline tokens are accepted in the following places (note
that a semicolon in place of the newline would be illegal in every one
of these cases):

- between the condition of a 
  [conditional expression](#conditional-expressions)
  or [while loop](#while-loop-expressions) and the next
  following expression,
- between the enumerators of a 
  [for-comprehension](#for-comprehensions-and-for-loops)
  and the next following expression, and
- after the initial `type`{.scala} keyword in a 
  [type definition or declaration](#type-declarations-and-type-aliases).

A single new line token is accepted

- in front of an opening brace ‘{’, if that brace is a legal
  continuation of the current statement or expression,
- after an [infix operator](#prefix-infix-and-postfix-operations), 
  if the first token on the next line can start an expression,
- in front of a [parameter clause](#function-declarations-and-definitions), and
- after an [annotation](#user-defined-annotations).

(@) The following code contains four well-formed statements, each
    on two lines. The newline tokens between the two lines are not
    treated as statement separators.

    ~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    if (x > 0)
      x = x - 1

    while (x > 0)
      x  = x / 2

    for (x <- 1 to 10)
      println(x)

    type
      IntList = List[Int]
    ~~~~~~~~~~~~~~~~~~~~~~

(@) The following code designates an anonymous class:

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    new Iterator[Int]
    {
      private var x = 0
      def hasNext = true
      def next = { x += 1; x }
    }
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~

    With an additional newline character, the same code is interpreted as
    an object creation followed by a local block:

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    new Iterator[Int] 

    {
      private var x = 0
      def hasNext = true
      def next = { x += 1; x }
    }
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~

(@) The following code designates a single expression:

    ~~~~~~~~~~~~ {.scala}
      x < 0 ||
      x > 10
    ~~~~~~~~~~~~

    With an additional newline character, the same code is interpreted as
    two expressions:

    ~~~~~~~~~~~ {.scala}
      x < 0 ||

      x > 10
    ~~~~~~~~~~~

(@) The following code designates a single, curried function definition:

    ~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    def func(x: Int)
            (y: Int) = x + y
    ~~~~~~~~~~~~~~~~~~~~~~~~~

    With an additional newline character, the same code is interpreted as
    an abstract function definition and a syntactically illegal statement:

    ~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    def func(x: Int)

            (y: Int) = x + y
    ~~~~~~~~~~~~~~~~~~~~~~~~~

(@) The following code designates an attributed definition:

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    @serializable
    protected class Data { ... }
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    With an additional newline character, the same code is interpreted as
    an attribute and a separate statement (which is syntactically
    illegal).

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    @serializable

    protected class Data { ... }
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Literals
----------

There are literals for integer numbers, floating point numbers,
characters, booleans, symbols, strings.  The syntax of these literals is in
each case as in Java.

<!-- TODO 
  say that we take values from Java, give examples of some lits in
  particular float and double. 
-->

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
Literal  ::=  [‘-’] integerLiteral
           |  [‘-’] floatingPointLiteral
           |  booleanLiteral
           |  characterLiteral
           |  stringLiteral
           |  symbolLiteral
           |  ‘null’
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


### Integer Literals

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
integerLiteral  ::=  (decimalNumeral | hexNumeral | octalNumeral) 
                       [‘L’ | ‘l’]
decimalNumeral  ::=  ‘0’ | nonZeroDigit {digit}
hexNumeral      ::=  ‘0’ ‘x’ hexDigit {hexDigit}
octalNumeral    ::=  ‘0’ octalDigit {octalDigit}
digit           ::=  ‘0’ | nonZeroDigit
nonZeroDigit    ::=  ‘1’ | … | ‘9’
octalDigit      ::=  ‘0’ | … | ‘7’
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Integer literals are usually of type `Int`{.scala}, or of type
`Long`{.scala} when followed by a `L` or
`l` suffix. Values of type `Int`{.scala} are all integer
numbers between $-2^{31}$ and $2^{31}-1$, inclusive.  Values of
type `Long`{.scala} are all integer numbers between $-2^{63}$ and
$2^{63}-1$, inclusive. A compile-time error occurs if an integer literal
denotes a number outside these ranges.

However, if the expected type [_pt_](#expression-typing) of a literal
in an expression is either `Byte`{.scala}, `Short`{.scala}, or `Char`{.scala}
and the integer number fits in the numeric range defined by the type,
then the number is converted to type _pt_ and the literal's type
is _pt_. The numeric ranges given by these types are:

---------------  -----------------------
`Byte`{.scala}   $-2^7$ to $2^7-1$
`Short`{.scala}  $-2^{15}$ to $2^{15}-1$
`Char`{.scala}   $0$ to $2^{16}-1$
---------------  -----------------------

(@) Here are some integer literals:

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    0          21          0xFFFFFFFF       0777L
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


### Floating Point Literals

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
floatingPointLiteral  ::=  digit {digit} ‘.’ {digit} [exponentPart] [floatType]
                        |  ‘.’ digit {digit} [exponentPart] [floatType]
                        |  digit {digit} exponentPart [floatType]
                        |  digit {digit} [exponentPart] floatType
exponentPart          ::=  (‘E’ | ‘e’) [‘+’ | ‘-’] digit {digit}
floatType             ::=  ‘F’ | ‘f’ | ‘D’ | ‘d’
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Floating point literals are of type `Float`{.scala} when followed by
a floating point type suffix `F` or `f`, and are
of type `Double`{.scala} otherwise.  The type `Float`{.scala}
consists of all IEEE 754 32-bit single-precision binary floating point
values, whereas the type `Double`{.scala} consists of all IEEE 754
64-bit double-precision binary floating point values.

If a floating point literal in a program is followed by a token
starting with a letter, there must be at least one intervening
whitespace character between the two tokens.

(@) Here are some floating point literals:

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    0.0        1e30f      3.14159f      1.0e-100      .1
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

(@) The phrase `1.toString`{.scala} parses as three different tokens:
    `1`{.scala}, `.`{.scala}, and `toString`{.scala}. On the
    other hand, if a space is inserted after the period, the phrase
    `1. toString`{.scala} parses as the floating point literal
    `1.`{.scala} followed by the identifier `toString`{.scala}.


### Boolean Literals

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
booleanLiteral  ::=  ‘true’ | ‘false’
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The boolean literals `true`{.scala} and `false`{.scala} are
members of type `Boolean`{.scala}.


### Character Literals

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
characterLiteral  ::=  ‘'’ printableChar ‘'’
                    |  ‘'’ charEscapeSeq ‘'’
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A character literal is a single character enclosed in quotes.
The character is either a printable unicode character or is described
by an [escape sequence](#escape-sequences).

(@) Here are some character literals:

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    'a'    '\u0041'    '\n'    '\t'
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Note that `'\u000A'` is _not_ a valid character literal because
Unicode conversion is done before literal parsing and the Unicode
character \\u000A (line feed) is not a printable
character. One can use instead the escape sequence `'\n'` or
the octal escape `'\12'` ([see here](#escape-sequences)).


### String Literals

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
stringLiteral  ::=  ‘\"’ {stringElement} ‘\"’
stringElement  ::=  printableCharNoDoubleQuote  |  charEscapeSeq
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A string literal is a sequence of characters in double quotes.  The
characters are either printable unicode character or are described by
[escape sequences](#escape-sequences). If the string literal
contains a double quote character, it must be escaped,
i.e. `"\""`. The value of a string literal is an instance of
class `String`{.scala}. 

(@) Here are some string literals:

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    "Hello,\nWorld!"       
    "This string contains a \" character."
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#### Multi-Line String Literals

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
stringLiteral   ::=  ‘"""’ multiLineChars ‘"""’
multiLineChars  ::=  {[‘"’] [‘"’] charNoDoubleQuote} {‘"’}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A multi-line string literal is a sequence of characters enclosed in
triple quotes `""" ... """`{.scala}. The sequence of characters is
arbitrary, except that it may contain three or more consuctive quote characters
only at the very end. Characters
must not necessarily be printable; newlines or other
control characters are also permitted.  Unicode escapes work as everywhere else, but none
of the escape sequences [here](#escape-sequences) are interpreted.

(@) Here is a multi-line string literal:

    ~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
      """the present string
         spans three 
         lines."""
    ~~~~~~~~~~~~~~~~~~~~~~~~

    This would produce the string:

    ~~~~~~~~~~~~~~~~~~~
    the present string
         spans three 
         lines.
    ~~~~~~~~~~~~~~~~~~~

The Scala library contains a utility method `stripMargin`
which can be used to strip leading whitespace from multi-line strings.
The expression

~~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
 """the present string
    spans three 
    lines.""".stripMargin
~~~~~~~~~~~~~~~~~~~~~~~~~~

evaluates to

~~~~~~~~~~~~~~~~~~~~ {.scala}
the present string
spans three 
lines.
~~~~~~~~~~~~~~~~~~~~

Method `stripMargin` is defined in class
[scala.collection.immutable.StringLike](http://www.scala-lang.org/api/current/index.html#scala.collection.immutable.StringLike). 
Because there is a predefined
[implicit conversion](#implicit-conversions) from `String`{.scala} to
`StringLike`{.scala}, the method is applicable to all strings.


### Escape Sequences

The following escape sequences are recognized in character and string
literals.

------  ------------------------------
`\b`    `\u0008`: backspace BS
`\t`    `\u0009`: horizontal tab HT
`\n`    `\u000a`: linefeed LF
`\f`    `\u000c`: form feed FF
`\r`    `\u000d`: carriage return CR
`\"`    `\u0022`: double quote "
`\'`    `\u0027`: single quote '
`\\`    `\u005c`: backslash `\`
------  -------------------------------

A character with Unicode between 0 and 255 may also be represented by
an octal escape, i.e. a backslash ‘\’ followed by a
sequence of up to three octal characters.

It is a compile time error if a backslash character in a character or
string literal does not start a valid escape sequence.


### Symbol literals

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
symbolLiteral  ::=  ‘'’ plainid
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A symbol literal `'x`{.scala} is a shorthand for the expression
`scala.Symbol("x")`{.scala}. `Symbol` is a [case class](#case-classes), 
which is defined as follows.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
package scala
final case class Symbol private (name: String) {
  override def toString: String = "'" + name
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The `apply`{.scala} method of `Symbol`{.scala}'s companion object
caches weak references to `Symbol`{.scala}s, thus ensuring that
identical symbol literals are equivalent with respect to reference
equality.


Whitespace and Comments
-----------------------

Tokens may be separated by whitespace characters
and/or comments. Comments come in two forms:

A single-line comment is a sequence of characters which starts with
`//` and extends to the end of the line.

A multi-line comment is a sequence of characters between
`/*` and `*/`. Multi-line comments may be nested,
but are required to be properly nested.  Therefore, a comment like
`/* /* */` will be rejected as having an unterminated
comment.


XML mode
--------

In order to allow literal inclusion of XML fragments, lexical analysis
switches from Scala mode to XML mode when encountering an opening
angle bracket '<' in the following circumstance: The '<' must be
preceded either by whitespace, an opening parenthesis or an opening
brace and immediately followed by a character starting an XML name.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.grammar}
 ( whitespace | ‘(’ | ‘{’ ) ‘<’ (XNameStart | ‘!’ | ‘?’)

  XNameStart ::= ‘_’ | BaseChar | Ideographic // as in W3C XML, but without ‘:’
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The scanner switches from XML mode to Scala mode if either

- the XML expression or the XML pattern started by the initial ‘<’ has been 
  successfully parsed, or if
- the parser encounters an embedded Scala expression or pattern and 
  forces the Scanner 
  back to normal mode, until the Scala expression or pattern is
  successfully parsed. In this case, since code and XML fragments can be
  nested, the parser has to maintain a stack that reflects the nesting
  of XML and Scala expressions adequately.

Note that no Scala tokens are constructed in XML mode, and that comments are interpreted
as text.

(@) The following value definition uses an XML literal with two embedded
Scala expressions

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ {.scala}
    val b = <book>
              <title>The Scala Language Specification</title>
              <version>{scalaBook.version}</version>
              <authors>{scalaBook.authors.mkList("", ", ", "")}</authors>
            </book>
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~