LINQ: Language Integrated Query. ST Colloquium, 2008-05-15 Tom Lokhorst

LINQ: Language Integrated Query ST Colloquium, 2008-05-15 Tom Lokhorst

Brief example

Brief example int[] nrs = {2, 3, 9, 1, 21, 3, 42};

Brief example int[] nrs = {2, 3, 9, 1, 21, 3, 42}; var q = from n in nrs where n < 5 select n * n;

Brief example int[] nrs = {2, 3, 9, 1, 21, 3, 42}; var q = from n in nrs where n < 5 select n * n; foreach (var i in q) { Console.WriteLine(i); }

Brief example int[] nrs = {2, 3, 9, 1, 21, 3, 42}; var q = from n in nrs where n < 5 select n * n; LINQ foreach (var i in q) { Console.WriteLine(i); }

Overview The problem space LINQ in.net 3.5 Deconstructing expressions Usages of LINQ Comparisons with other stuff

Circles, Triangles and Rectangles

Circles, Triangles and Rectangles

Circles, Triangles and Rectangles Business software deals with lots of different types of data: Objects Tree structures (XML) Relational

Assignment A web server should show a list of the top 5 memory-intensive processes. Per process, show its name, and a description. There is a SQL database with descriptions for programs. The list should be in RSS format.

A bit of History

A bit of History.NET 1.0 released early 2002 CLR 1.0, C# 1.0, VB 7.0 Delegates: managed function pointers

A bit of History.NET 1.0 released early 2002 CLR 1.0, C# 1.0, VB 7.0 Delegates: managed function pointers.net 2.0 released late 2005 CLR 2.0, C# 2.0, VB 8.0 Generics, iterators, anonymous delegates

A bit of History.NET 1.0 released early 2002 CLR 1.0, C# 1.0, VB 7.0 Delegates: managed function pointers.net 2.0 released late 2005 CLR 2.0, C# 2.0, VB 8.0 Generics, iterators, anonymous delegates.net 3.5 released late 2007 CLR 2.0, C# 3.0, VB 9.0 Lambda expressions, local type inferencing, extension methods, LINQ

LINQ in.net 3.5 Libraries LINQ to Objects LINQ to XML LINQ to SQL Language enhancements Lambda expressions Query syntax Compiler enhancements Expression trees

LINQ to Objects

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 var q = from n in nrs where n < 5 select n * n;

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 IEnumerable<int> q = from n in nrs where n < 5 select n * n;

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 IEnumerable<int> q = nrs.where<int>(n => n < 5).Select<int, int>(n => n * n);

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 IEnumerable<float> q = nrs.where<int>(n => n < 5).Select<int, float>(n => n * 0.5F);

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 IEnumerable<float> q = Enumerable.Select<int, float>( Enumerable.Where<int>(nrs, n => n < 5), (n => n * 0.5F));

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 IEnumerable<float> q = Enumerable.Select<int, float>( Enumerable.Where<int>(nrs, n => n < 5), (n => n * 0.5F)); public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Func<T, bool> predicate) {... }

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 IEnumerable<float> q = Enumerable.Select<int, float>( Enumerable.Where<int>(nrs, n => n < 5), (n => n * 0.5F)); public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Func<T, bool> predicate) { foreach (var elem in source) if (predicate(elem)) yield return elem; }

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 IEnumerable<float> q = Enumerable.Select<int, float>( Enumerable.Where<int>(nrs, n => n < 5), (n => n * 0.5F));

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 2.0 IEnumerable<float> q = Enumerable.Select<int, float>( Enumerable.Where<int>(nrs, delegate(int n){ return n < 5; }), delegate(int n){ return n * 0.5F;} );

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 1.0* IEnumerable<float> q = Enumerable.Select<int, float>( Enumerable.Where<int>(nrs, new Func<int, bool>(lessthanfive)), new Func<int, float>(timeshalf) ); bool LessThanFive(int n) { return n < 5; } float TimesHalf(int n) { return n * 0.5F; }

Eating the sugar int[] nrs = {2, 3, 9, 1, 21, 3, 42}; C# 3.0 var q = from n in nrs where n < 5 select n * n;

LINQ to Objects Query comprehenssions More syntax; orderby, groupby, join Works on you data structures IEnumerable<T> Implement you own Where<T>, Select<T> More library functions

LINQ to XML

Dealing with angle brackets LINQ to XML is a replacement for the W3C Document Object Model, because: It s old It s ugly It s impractical New API, all library: System.Xml.LINQ

Document Object Model Imperative API Document-centric Weird data types

Document Object Model Imperative API Document-centric Weird data types <Company name= Microsoft > <Founders> <Person>Bill Gates</Person> </Founders> </Company>

Document Object Model XmlDocument doc = new XmlDocument(); XmlElement company = doc.createelement("company"); doc.appendchild(company); XmlAttribute name = doc.createattribute("name"); name.value = "Microsoft"; company.attributes.append(name); XmlElement founders = doc.createelement("founders"); company.appendchild(founders); XmlElement person = doc.createelement("person"); founders.appendchild(person); XmlText bill = doc.createtextnode("bill Gates"); person.appendchild(bill);

LINQ to XML Functional construction Element-centric (context free) CLR data types (collection friendly)

LINQ to XML Functional construction Element-centric (context free) CLR data types (collection friendly) XElement company = new XElement("Company", new XAttribute("name", "Microsoft"), new XElement("Founders", new XElement("Person", "Bill Gates") ) );

XML with queries

XML with queries string[] names = { Anders, Erik, Amanda };

XML with queries string[] names = { Anders, Erik, Amanda }; from n in names where n.startswith( A ) select new XElement( Person, n)

XML with queries string[] names = { Anders, Erik, Amanda }; XElement employees = new XElement( Employees, from n in names where n.startswith( A ) select new XElement( Person, n) );

XML with queries string[] names = { Anders, Erik, Amanda }; XElement employees = new XElement( Employees, from n in names where n.startswith( A ) select new XElement( Person, n) ); company.add(employees);

XML with queries string[] names = { Anders, Erik, Amanda }; XElement employees = new XElement( Employees, from n in names where n.startswith( A ) select new XElement( Person, n) ); company.add(employees); Console.WriteLine(company);

XML on the Console <Company name= Microsoft > <Founders> <Person>Bill Gates</Person> </Founders> <Employees> <Person>Anders</Person> <Person>Amanda</Person> </Employees> </Company>

Querying XML IEnumerable<string> persons = from p in company.descendants( Person ) select p.value;

Querying XML IEnumerable<string> persons = from p in company.descendants( Person ) select p.value; Bill Gates Anders Amanda

LINQ to SQL

Querying a SQL backend Use LINQ to query a relational database Strongly typed queries Remote the query to the DBMS C# must remain API/DB independent

LINQ to SQL Simple Object Relational Mapper Microsoft SQL Server Comes with tool to make classes out of tables in an existing database Certainly not the best ORM out there More advanced stuff: LINQ to Entities

LINQ to SQL example

LINQ to SQL example

LINQ to SQL example

LINQ to SQL example [DatabaseAttribute(Name="Northwind")] public partial class NorthwindDataContext : System.Data.LINQ.DataContext { public NorthwindDataContext() : base("data Source=.;Initial Catalog=Northwind;" + "Integrated Security=True") { } } public Table<Customer> Customers { get { return this.gettable<customer>(); } }

LINQ to SQL example [Table(Name="dbo.Customers")] public partial class Customer : INotifyPropertyChanging, INotifyPropertyChanged { private string _CustomerID; [Column(Storage="_CustomerID", DbType="NChar(5) NOT NULL", CanBeNull=false, IsPrimaryKey=true)] public string CustomerID { get { return this._customerid; } set { /* Setter code removed */ } } } /* More code be here */

LINQ to SQL example

LINQ to SQL example var db = new NorthwindDataContext(); var q = from c in db.customers where c.city == London select c;

LINQ to SQL example var db = new NorthwindDataContext(); var q = from c in db.customers where c.city == London select c; foreach (var c in q) Console.WriteLine(c.CustomerID + + c.contactname);

LINQ to SQL example var db = new NorthwindDataContext(); var q = from c in db.customers where c.city == London select c; foreach (var c in q) Console.WriteLine(c.CustomerID + + c.contactname); AROUT Thomas Hardy BSBEV Victoria Ashworth CONSH Elizabeth Brown EASTC Ann Devon NORTS Simon Crowther SEVES Hari Kumar

LINQ to SQL example var q = db.customers.where<customer>( c => c.city == London );

Lambdas again

Lambdas again Func<int, int> f = x => (x + 1) * 2; int nr = f(20); Console.WriteLine(nr); // Prints: 42

Lambdas again Expression<Func<int, int>> e = x => (x + 1) * 2;

Lambdas again Expression<Func<int, int>> e = x => (x + 1) * 2; string str = e.nodetype + + e.parameters.count; Console.WriteLine(str); // Prints: Lambda 1

Lambdas again Expression<Func<int, int>> e = x => (x + 1) * 2; string str = e.nodetype + + e.parameters.count; Console.WriteLine(str); // Prints: Lambda 1 Func<int, int> f = e.compile(); Console.WriteLine(f(20)); // Prints: 42

Lambdas again Expression<Func<int, int>> e = x => (x + 1) * 2;

Lambdas again Expression<Func<int, int>> e = x => (x + 1) * 2; ParameterExpression x = Expression.Parameter(typeof(int), "x"); Expression one = Expression.Constant(1, typeof(int)); Expression two = Expression.Constant(2, typeof(int)); Expression body = Expression.Multiply(Expression.Add(x, one), two); Expression<Func<int, int>> e = Expression.Lambda<Func<int, int>>(body, x);

LINQ to SQL example var q = db.customers.where<customer>( c => c.city == London );

LINQ to SQL example var q = db.customers.where<customer>( c => c.city == London ); ParameterExpression c = Expression.Parameter(typeof(Customer), "c"); IQueryable<Customer> q = db.customers.where<customer>(expression.lambda<func<customer, bool>>( Expression.Equal( Expression.Property(c, (MethodInfo) methodof(customer.get_city)), Expression.Constant("London", typeof(string)), false, (MethodInfo) methodof(string.op_equality)), c));

LINQ to SQL example var q = db.customers.where<customer>( c => c.city == London ); SELECT C.* FROM Customers AS C WHERE C.City = London

Many more operations int x = db.customers.where<customer>( c => c.city == London ).Count(); SELECT COUNT(C.*) AS value FROM Customers AS C WHERE C.City = London

Many more operations int x = db.customers.where<customer>( c => c.city == London ).Take(4); SELECT TOP (4) C.* FROM Customers AS C WHERE C.City = London

Many more operations Restriction Projection Partitioning Join Concatinatin Ordering Grouping Set Conversion Elemement Generation Quantifiers Aggregate Where Select, SelectMany Take, Skip, TakeWhile, SkipWhile Join, GroupJoin Concat OrderBy, ThenBy, Reverse GroupBy Distinct, Union, Intersect, Except ToArray, ToList, ToDictionary, OfType, Cast First, FirstOrDefault, Last, LastOrDefault Range, Repeat, Empty Any, All, Contains Count, Sum, Min, Max, Average, Aggregate

Assignment

Assignment Solution

Assignment Solution new XElement("rss", new XAttribute("version", "2.0"), new XElement("channel", new XElement("title", "Top Processes"), from p in (from p in Process.GetProcesses() orderby p.virtualmemorysize descending select p).take(5) join pd in db.processdescriptions on p.processname equals pd.name into descrs from d in descrs.defaultifempty() select new XElement("item", new XElement("title", p.processname), d!= null? new XElement("description", d.description) : null) ) );

Assignment <rss version="2.0"> <channel> <title>top Processes</title> <item> <title>sqlservr</title> <description>sql Server</description> </item> <item> <title>sqlservr</title> <description>sql Server</description> </item> <item> <title>devenv</title> <description>visual Studio</description> </item> <item> <title>ssmsee</title> </item> <item> <title>reflector</title> </item> </channel> </rss> Solution

Other providers DbLinq (MySQL, PostgreSQL, Oracle) LINQ to Google LINQ to Entities Parallel LINQ DryadLINQ (Distributed execution engine)

Concluding... All LINQ is: Query comprehensions Libraries Expression trees Many more providers to come...

Questions?

Assignment in VB syntax Dim result = _ <rss version="2.0"> <channel>top Processes</channel> <%= From p In Process.GetProcesses() _ Order By p.virtualmemorysize Descending _ Take 5 _ Group Join pd In db.processdescriptions _ On p.processname Equals pd.name _ Into descrs = Group _ From d In descrs.defaultifempty _ Select <item> <title><%= p.processname %></title> <%= If(d IsNot Nothing, _ <description><%= d.description %></description>, _ Nothing) %> </item> %> </rss>

LINQ in Java // Setup a JPA entity manager... SessionFactory sessionfactory = new Configuration().configure().buildSessionFactory(); EntityManagerFactory entitymanagerfactory = new EntityManagerFactoryImpl(sessionFactory, PersistenceUnitTransactionType.RESOURCE_LOCAL, true); QueryableEntityManager entitymanager = new QueryableEntityManager(entityManagerFactory.createEntityManager()); // Select all customers in the Washington region Iterable<Customer> wacustomers = from("c").in(entitymanager.entity(customer.class)). where(eq("c.getregion()", "WA")). select("c");

Monadic parser combinators using C# public abstract class MiniMLParsers<TInput> : CharParsers<TInput>{ public MiniMLParsers() { Whitespace = Rep(Char(' ').OR(Char('\t').OR(Char('\n')).OR(Char('\r')))); WsChr = chr => Whitespace.AND(Char(chr)); Id = from w in Whitespace from c in Char(char.IsLetter) from cs in Rep(Char(char.IsLetterOrDigit)) select cs.aggregate(c.tostring(),(acc,ch) => acc+ch); Ident = from s in Id where s!= "let" && s!= "in" select s; LetId = from s in Id where s == "let" select s; InId = from s in Id where s == "in" select s; Term1 = (from x in Ident select (Term)new VarTerm(x)).OR( (from u1 in WsChr('(') from t in Term from u2 in WsChr(')') select t)); Term = (from u1 in WsChr('\\') from x in Ident from u2 in WsChr('.') from t in Term

} from cs in Rep(Char(char.IsLetterOrDigit)) select cs.aggregate(c.tostring(),(acc,ch) => acc+ch); Ident = from s in Id where s!= "let" && s!= "in" select s; LetId = from s in Id where s == "let" select s; Monadic parser InId = from s in Id where s == "in" select s; Term1 = (from x in Ident select (Term)new VarTerm(x)) combinators using C#.OR( (from u1 in WsChr('(') from t in Term from u2 in WsChr(')') select t)); Term = (from u1 in WsChr('\\') from x in Ident from u2 in WsChr('.') from t in Term select (Term)new LambdaTerm(x,t)).OR( (from letid in LetId from x in Ident from u1 in WsChr('=') from t in Term from inid in InId from c in Term select (Term)new LetTerm(x,t,c))).OR( (from t in Term1 from ts in Rep(Term1) select (Term)new AppTerm(t,ts))); All = from t in Term from u in WsChr(';') select t;