All of the values we saw in the previous section were simple values. In M, a simple value is a value that has no uniform way to be decomposed into constituent parts. While there are textual operators that allow you to extract substrings from a text value, those operators are specific to textual data and don’t work on numeric data. Similarly, any “bit-level” operations on binary values don’t apply to text or numeric data.

An M collection is a value that groups together zero or more elements that themselves are values. We can write down collections in expressions using an initializer, { }.

The following expressions each use an initializer to yield a collection value:

{ 1, 2 }
{ 1 }
{ }

As with simple values, the equivalence operators == and != are defined over collections. In M, two collections are considered equivalent if and only if each element has a distinct equivalent element in the other collection. That allows us to write the following equivalence expressions:

{ 1, 2 } == { 1, 2 }
{ 1, 2 } != { 1 }

both of which are true.

The elements of a collection can consist of different kinds of values:

{ true, "Hello" }

and these values can be the result of arbitrary calculation:

{ 1 + 2, 99 – 3, 4 < 9 }

which is equivalent to the collection:

{ 3, 96, true }.

The order of elements in a collection is not significant. That means that the following expression is also true:

{ 1, 2 } == { 2, 1 }

Finally, collections can contain duplicate elements, which are significant. That makes the following expression:

{ 1, 2, 2 } != { 1, 2 }

also true.

M defines a set of built-in operators that are specific to collections. The most important is the in operator, which tests whether a given value is an element of the collection. The result of the in operator is a logical value that indicates whether the value is or is not an element of the collection. For example, these expressions:

1 in { 1, 2, 3 }
!(1 in { "Hello", 9 })

both result in true.

M defines a Count member on collections that calculates the number of elements in a collection. This use of that operator:

{ 1, 2, 2, 3 }.Count

results in the value 4. The postfix # operator returns the count of a collection, so

{ 1, 2, 2, 3 }# == { 1, 2, 2, 3 }.Count

returns true.

As noted earlier, M collections may contain duplicates. You can apply the Distinct member to get a version of the collection with any duplicates removed:

{ 1, 2, 3, 1 }.Distinct == { 1, 2, 3 }

The result of Distinct is not just a collection but is also a set, that is, a collection of distinct elements.

M also defines set union "|" and set intersection "&" operators, which also yield sets:

({ 1, 2, 3, 1 } | { 1, 2, 4 }) == { 1, 2, 3, 4 }
({ 1, 2, 3, 1 } & { 1, 2, 4 }) == { 1, 2 }

Note that union and intersection always return collections that are sets, even when applied to collections that contain duplicates.

M defines the subset and superset using <= and >=. Again these operations convert collections to sets. The following expressions evaluate to true.

{ 1, 2 } <= { 1, 2, 3 }
{ "Hello", "World" } >= { "World" }
{ 1, 2, 1 } <= { 1, 2, 3 }

Arguably the most commonly used collection operator is the where operator. The where operator applies a logical expression (called the predicate) to each element in a collection (called the domain) and results in a new collection that consists of only the elements for which the predicate holds true. To allow the element to be used in the predicate, the where operator introduces the symbol value to stand in for the specific element being tested.

For example, consider this expression that uses a where operator:

{ 1, 2, 3, 4, 5, 6 } where value > 3

In this example, the domain is the collection { 1, 2, 3, 4, 5, 6 } and the predicate is the expression value > 3. Note that the identifier value is available only within the scope of the predicate expression. The result of this expression is the collection { 4, 5, 6 }.

M supports a richer set of query comprehensions using a syntax similar to that of Language Integrated Query (LINQ). For example, the where example just shown can be written in long form as follows:

from value in { 1, 2, 3, 4, 5, 6 }
where value > 3
select value

In general, M supports the LINQ operators with these significant exceptions:

While the where operator allows elements to be accessed based on a calculation over the values of each element, there are situations where it would be much more convenient to simply assign names to each element and then access the element values by its assigned name. M defines a distinct kind of value called an entity for just this purpose.

An entity consists of zero or more name-value pairs called fields. Entities can be constructed in M using an initializer. Here’s a simple entity value:

{ X = 100, Y = 200 }

This entity has two fields: one named X with the value of 100, the other named Y with the value of 200.

Entity initializers can use arbitrary expressions as field values:

{ X = 50 + 50, Y = 300 - 100 }

And the names of members can be arbitrary Unicode text:

{
  [Horizontal Coordinate] = 100,
  [Vertical Coordinate] = 200
}

If the member name matches the Identifier pattern, it can be written without the surrounding [ ]. An identifier must begin with an upper or lowercase letter or "_" and be followed by a sequence of letters, digits, "_", and "$".

Here are a few examples:

HelloWorld = 1     // matches the Identifier pattern
[Hello World] = 1  // doesn't match identifier pattern
_HelloWorld = 1    // matches the Identifier pattern
A = 1              // matches the Identifier pattern
[1] = 1            // doesn't match identifier pattern

It is always legal to use [ ] to escape symbolic names; however, most of the examples in this document use names that don’t require escaping and, therefore, do not use escaping syntax for readability.

M imposes no limitations on the values of entity members. It is legal for the value of an entity member to refer to another entity:

{
  TopLeft = { X = 100, Y = 200 },
  BottomRight = { X = 400, Y = 100 }
}

or a collection:

{
  LotteryPicks = { 1, 18, 25, 32, 55, 61 },
  Odds = 0.00000001
}

or a collection of entities:

{
  Color = "Red",
  Path = {
    { X = 100, Y = 100 },
    { X = 200, Y = 200 },
    { X = 300, Y = 100 },
    { X = 300, Y = 100 },
  }
}

This last example illustrates that entity values are legal for use as elements in collections.

Entity initializers are useful for constructing new entity values. M defines the dot, “.”, operator over entities for accessing the value of a given member. For example, this expression:

{ X = 100, Y = 200 }.X

yields the value of the X member, which in this case is 100. The result of the dot operator is just a value that is subject to subsequent operations. For example, this expression:

{ Center = { X = 100, Y = 200 }, Radius = 3 }.Center.Y

yields the value 200.

M defines a type constructor for specifying collection types. The collection type constructor restricts the type and count of elements a collection may contain. All collection types are restrictions over the intrinsic type Collection, which all collection values conform to:

{ } in Collection
{ 1, false } in Collection
! ("Hello" in Collection)

The last example is interesting, in that it illustrates that the collection types do not overlap with the simple types. There is no value that conforms to both a collection type and a simple type.

A collection type constructor specifies both the type of element and the acceptable element count. The element count is typically specified using one of the three operators:

T* – zero or more Ts
T+ – one or more Ts
T#m..n – between m and n Ts

The collection type constructors can either use operators or be written longhand as a constraint over the intrinsic type Collection:

type SomeNumbers : Number+;
type TwoToFourNumbers : Number#2..4;
type ThreeNumbers : Number#3;
type FourOrMoreNumbers : Number#4..;

These types describe the same sets of values as these longhand definitions:

type SomeNumbers : Collection where value.Count >= 1
                                    && item in Number;
type TwoToFourNumbers : Collection where value.Count >= 2
                                    && value.Count <= 4
                                    && item in Number;
type ThreeNumbers : Collection where value.Count == 3
                                    && item in Number;
type FourOrMoreNumbers : Collection where value.Count >= 4
                                    && item in Number;

In the case that value itself is a collection, an additional variable item is introduced into scope. The item variable ranges over the elements of value (which must be a collection). Clauses that use item must hold for every element of value.

Independent of which form is used to declare the types, we can now assert the following hold:

!({ } in TwoToFourNumbers)
!({ "One", "Two", "Three" } in TwoToFourNumbers)
{ 1, 2, 3 } in TwoToFourNumbers
{ 1, 2, 3 } in ThreeNumbers
{ 1, 2, 3, 4, 5 } in FourOrMoreNumbers

The collection type constructors compose with the where operator, allowing the following type check to succeed:

{ 1, 2 } in (Number where value < 3)*
  where value.Count % 2 == 0

Note that the where inside the parentheses applies to elements of the collection, and the where outside the parentheses operator applies to the collection itself.

We have seen many useful values: 42, "Hello", {1,2,3}. The distinguished value null serves as a placeholder for some other value that is not known. A type with null in the value space is called a nullable type. The value null can be added to the value space of a type with an explicit union of the type and a collection containing null or using the postfix operator ?. The following expressions are true:

! (null in Integer)
null in Integer?
null in (Integer | { null } )

The ?? operator converts between a null value and known value:

null ?? 1 == 1
3 ?? 1 == 3

Arithmetic operations on a null operand return null:

1 + null == null
null * 3 == null

Logical operators, conditional, and constraints require non-nullable operands.

Just as we can use the collection type constructors to specify what kinds of collections are valid in a given context, we can do the same for entities using entity types.

An entity type declares the expected members for a set of entity values. The members of an entity type can be declared either as fields or as computed values. The value of a field is stored; a computed value is evaluated. All entity types are restrictions over the Entity type.

Here is the simplest entity type:

type MyEntity : Language.Entity;

The type MyEntity does not declare any fields. In M, entity types are open in that entity values that conform to the type may contain fields whose names are not declared in the type. That means that the following type test:

{ X = 100, Y = 200 } in MyEntity

will evaluate to true, as the MyEntity type says nothing about fields named X and Y.

Most entity types contain one or more field declarations. At a minimum, a field declaration states the name of the expected field:

type Point { X; Y; }

This type definition describes the set of entities that contain at least fields named X and Y irrespective of the values of those fields. That means that the following type tests will all evaluate to true:

{ X = 100, Y = 200 } in Point

// more fields than expected OK
{ X = 100, Y = 200, Z = 300 } in Point

// not enough fields – not OK
! ({ X = 100 } in Point)

{ X = true, Y = "Hello, world" } in Point

The last example demonstrates that the Point type does not constrain the values of the X and Y fields—any value is allowed. We can write a new type that constrains the values of X and Y to numeric values:

type NumericPoint {
  X : Number;
  Y : Number where value > 0;
}

Note that we’re using type ascription syntax to assert that the value of the X and Y fields must conform to the type Number. With this in place, the following expressions all evaluate to true:

{ X = 100, Y = 200 } in NumericPoint
{ X = 100, Y = 200, Z = 300 } in NumericPoint
! ({ X = true, Y = "Hello, world" } in NumericPoint)
! ({ X = 0, Y = 0 } in NumericPoint)

As we saw in the discussion of simple types, the name of the type exists only so that M declarations and expressions can refer to it. That is why both of the following type tests succeed:

{ X = 100, Y = 200 } in NumericPoint
{ X = 100, Y = 200 } in Point

even though the definitions of NumericPoint and Point are independent.

Fields are named units of storage that hold values. M allows you to initialize the value of a field as part of an entity initializer. However, M does not specify any mechanism for changing the value of a field once it is initialized. In M, we assume that any changes to field values happen outside the scope of M.

A field declaration can indicate that there is a default value for the field. Field declarations that have a default value do not require conformant entities to have a corresponding field specified. (We sometimes call such field declarations optional fields.) For example, consider this type definition:

type Point3d {
  X : Number;
  Y : Number;
  Z = -1 : Number; // default value of negative one
}

Because the Z field has a default value, the following type test will succeed:

{ X = 100, Y = 200 } in Point3d

Moreover, if we apply a type ascription operator to the value:

({ X = 100, Y = 200 } : Point3d)

we can now access the Z field like this:

({ X = 100, Y = 200 } : Point3d).Z

This expression will yield the value -1.

If a field declaration does not have a corresponding default value, conformant entities must specify a value for that field. Default values are typically written down using the explicit syntax shown for the Z field of Point3d. If the type of a field is either nullable or a zero-to-many collection, then there is an implicit default value for the declaring field of null for optional and {} for the collection.

For example, consider this type:

type PointND {
  X : Number;
  Y : Number;
  Z : Number?;        // Z is optional
  BeyondZ : Number*;  // BeyondZ is optional too
}

Again, the following type test will succeed:

{ X = 100, Y = 200 } in PointND

and ascribing the PointND to the value will allow us to get these defaults:

({ X = 100, Y = 200 } : PointND).Z == null
({ X = 100, Y = 200 } : PointND).BeyondZ == { }

The choice of using a nullable type versus an explicit default value to model optional fields typically comes down to style.

Calculated values are named expressions whose values are computed rather than stored. Here’s an example of a type that declares a computed value, IsHigh:

type PointPlus {
  X : Number;

  Y : Number;

// a computed value

  IsHigh() : Logical { Y > 0 }

}

Note that unlike field declarations which end in a semicolon, computed value declarations end with the expression surrounded by braces.

Like field declarations, a computed value declaration may omit the type ascription, as this example does:

type PointPlus {
  X : Number;
  Y : Number;
// a computed value with no type ascription
  InMagicQuadrant() { IsHigh && X > 0 }
  IsHigh() : Logical { Y > 0 }
}

When no type is explicitly ascribed to a computed value, M will infer the type automatically based on the declared result type of the underlying expression. In this example, because the logical-and operator used in the expression was declared as returning a Logical, the InMagicQuadrant computed value also is ascribed to yield a Logical value.

The two computed values we just defined and used didn’t require any additional information to calculate their results other than the entity value itself. A computed value may optionally declare a list of named parameters whose actual values must be specified when using the computed value in an expression. Here’s an example of a computed value that requires parameters:

type PointPlus {
  X : Number;
  Y : Number;
  // a computed value that requires a parameter
  WithinBounds(radius : Number) : Logical {
    X * X + Y * Y <= radius * radius
  }
  InMagicQuadrant() { IsHigh && X > 0 }
  IsHigh() : Logical { Y > 0 }
}

To use this computed value in an expression, you must provide values for the parameters:

({ X = 100, Y = 200 } : PointPlus).WithinBounds(50)

When calculating the value of WithinBounds, M will bind the value 50 to the symbol radius—this will cause the WithinBounds computed value to evaluate to false.

It is useful to note that both computed values and default values for fields are part of the type definition, not part of the values that conform to the type. For example, consider these three type definitions:

type Point {
  X : Number;
  Y : Number;
}
type RichPoint {
  X : Number;
  Y : Number;
  Z = -1 : Number;
  IsHigh() : Logical { X < Y }
}
type WeirdPoint {
  X : Number;
  Y : Number;
  Z = 42 : Number;
  IsHigh() : Logical { false }
}

Because RichPoint and WeirdPoint have only two required fields (X and Y), we can state the following:

{ X=1, Y=2 } in RichPoint
{ X=1, Y=2 } in WeirdPoint

However, the IsHigh computed value is only available when we ascribe one of these two types to the entity value:

({ X=1, Y=2 } : RichPoint).IsHigh == true
({ X=1, Y=2 } : WeirdPoint).IsHigh == false

Because IsHigh is purely part of the type and not the value, when we chain the ascription like this:

(({ X=1, Y=2 } : RichPoint) : WeirdPoint).IsHigh == false

the outer-most ascription that determines which function is called.

A similar principle is at play with respect to how default values work. Again, the default value is part of the type, not the entity value. When we write the following expression:

({ X=1, Y=2 } : RichPoint).Z == -1

the underlying entity value still only contains two field values (1 and 2 for X and Y respectively). Where default values differ from computed values is when we chain ascriptions. Consider this expression:

(({ X=1, Y=2 } : RichPoint) : WeirdPoint).Z == -1

Because the RichPoint ascription is applied first, the resultant entity has a field named Z whose value is -1; however, there is no storage allocated for the value. (It’s part of the type’s interpretation of the value.) When we apply the WeirdPoint ascription, we’re applying it to the result of the first ascription, which does have a field named Z, so that value is used to specify the value for Z—the default value specified by WeirdPoint is not needed.

Like all types, a constraint may be applied to an entity type using the where operator. Consider the following type definition:

type HighPoint {
  X : Number;
  Y : Number;
} where X < Y;

In this example, all values that conform to the type HighPoint are guaranteed to have an X value that is less than the Y value. That means that the following expressions:

{ X = 100, Y = 200 } in HighPoint
! ({ X = 300, Y = 200 } in HighPoint)

both evaluate to true.

Now consider the following type definitions:

type Point {
  X : Number;
  Y : Number;
}
type Visual {
  Opacity : Number;
}
type VisualPoint {
  DotSize : Number;
} where value in Point && value in Visual;

The third type, VisualPoint, names the set of entity values that have at least the numeric fields X, Y, Opacity, and DotSize.

Because it is a common desire to factor member declarations into smaller pieces that can be easily composed, M provides explicit syntax support for this. We can rewrite the VisualPoint type definition using that syntax:

type VisualPoint : Point, Visual {
  DotSize : Number;
}

To be clear, this is just shorthand for the preceding long-hand definition that used a constraint expression. Both of these definitions are equivalent to this even longer-hand definition:

type VisualPoint {
  X : Number;
  Y : Number;
  Opacity : Number;
  DotSize : Number;
}

Again, the names of the types are just ways to refer to types—the values themselves have no record of the type names used to describe them.

Filtering extracts elements from an existing collection. Consider the following collection:

People {
  { First = "Mary", Last = "Smith", Age = 24 },
  { First = "John", Last = "Doe", Age = 32 },
  { First = "Dave", Last = "Smith", Age = 32 },
}

This query extracts people with Age == 32 from the People collection:

from p in People
where p.Age == 32
select p

An equivalent query can be written with either of the following expressions:

People where value.Age == 32
People.Age(32)

The where operator takes a collection on the left and a Logical expression on the right. The where operator introduces a keyword identifier value into the scope of the Logical expression that is bound to each member of the collection. The resulting collection contains the members for which the expression is true. The expression:

Collection where Expression

is exactly equivalent to:

from value in Collection
where Expression
select value

Collection types gain indexer members that correspond to the fields of their corresponding element type. That is, this:

Collection . Field ( Expression )

is equivalent to:

from value in Collection
where Field == Expression
select value

Select is also available as an infix operator. Consider the following simple query:

from p in People
select p.First + p.Last

This computes the select expression over each member of the collection and returns the result. Using the infix select it can be written equivalently as:

People select value.First + value.Last

The select operator takes a collection on the left and an arbitrary expression on the right. As with where, select introduces the keyword identifier value that ranges over each element in the collection. The select operator maps the expression over each element in the collection and returns the result. The expression:

Collection select Expression

Is exactly equivalent to:

from value in Collection
select Expression

A trivial use of the select operator is to extract a single field:

People select value.First

Collections are augmented with accessors to fields that can be extracted directly. For example People.First yields a new collection containing all the first names, and People.Last yields a collection with all the last names.