EntityFramework-friendly computed properties

Putting together all of the nicest things from the latest dotnet

Oleksandr Redka

Published in

C# Programming

11 min readDec 29, 2023

The Goal

Let’s design a domain for a vacation request system. Here are a few key things we care about:

Each employee has a fixed vacation allowance — 25 vacation days in a given calendar year.
To avoid extra complexity we will assume that all employees are getting all 25 days at the beginning of the year.
Employees could request a vacation that could be pending, approved, or canceled.
Employee vacation budget is a difference between Vacation Allowance and total days of pending and approved vacation requests in a current year.

Here is the implementation following principles of the Rich Domain Model (More about it in my other article):

public class Employee
{
    public Guid Id { get; private set; }

    //Each employee has 25 vacation days in a given year.
    public int VacationAllowance { get; private set; } = 25;
    private readonly List<VacationRequest> vacationRequests = new();

    //Vacation budget is a difference between Vacation Allowance and total days of pending and approved vacation requests
    public double VacationBudget => VacationAllowance - VacationRequests
        .Where(request => request.StartTime.Year == DateTime.UtcNow.Year)
        .Where(request => request.State == VacationRequestState.Approved || request.State == VacationRequestState.Pending)
        .Sum(request => request.TotalDays);
    public IReadOnlyList<VacationRequest> VacationRequests => vacationRequests;

    public VacationRequest RequestVacation(DateTime startTime, DateTime endTime)
    {
        var vacationRequest = new VacationRequest(startTime, endTime);

        if (vacationRequest.TotalDays > VacationBudget)
        {
            throw new InvalidOperationException("Budget is exceeded!");
        }

        vacationRequests.Add(vacationRequest);

        return vacationRequest;
    }
}

public class VacationRequest
{
    public VacationRequest(Guid id, DateTime startTime, DateTime endTime, double totalDays, VacationRequestState state)
    {
        Id = id;
        StartTime = startTime;
        EndTime = endTime;
        TotalDays = totalDays;
        State = state;
    }

    public VacationRequest(DateTime startTime, DateTime endTime)
    {
        Id = Guid.NewGuid();
        StartTime = startTime;
        EndTime = endTime;
        TotalDays = (StartTime - EndTime).TotalDays;
        State = VacationRequestState.Pending;
    }
    public Guid Id { get; private set; }
    public DateTime StartTime { get; private set; }
    public DateTime EndTime { get; private set; }
    public double TotalDays { get; private set; }
    public VacationRequestState State { get; private set; }
}

public enum VacationRequestState
{
    Pending = 0,
    Approved = 1,
    Cancelled = 2
}

We will skip details on how Vacation Requests could be approved or canceled for now. The main thing we care about is the VacationBudget computed property.

Even though it’s nice and handy to have a computed property for the budget, EntityFramework will have no idea what to do with it. All projections with VacationBudget will result in whole Employee instance materialization. Let’s peek into the SQL that EF will generate for our case.

Here is our DbContext class. We will skip migrations, and re-create a database with each application run.

public class VacationSystemContext : DbContext
{
    public VacationSystemContext(DbContextOptions options) : base(options)
    {
    }

    public DbSet<Employee> Employees { get; }
}

Let’s run a simple query to get the list of employee IDs and respective VacationBudgets:

var query = dbContext.Employees.Select(employee => new
{
    employee.Id,
    employee.VacationBudget
}).ToQueryString();

And this is what EF will generate for us:

SELECT "e"."Id"
FROM "Employees" AS "e"

So, we can see that EF has no idea how to calculate VacationBudget property, so it will materialize all employees and perform Select on materialized entities. Regardless of EF’s efforts, the given query will eventually fail, as the VacationRequests table is not joined.

In this article, I want to create a way to “teach” Entity Framework to work with computed properties and translate them to SQL.

What do you need to know before we begin?

A decent understanding of Expression Trees is a great plus.
Differences between IEnumerable and IQueryable.

Action Plan

Relevant changes in EntityFramework

First, we can refer to one of the latest EntityFramework features: IQueryExpressionInterceptor.

A class that implements this interface will have access to ExpressionTree which is about to be compiled into a database query. Implementing this interceptor will allow us to find calls to our Computed properties and replace them with respective expression trees.

SourceGenerators

Another thing that was recently shipped is SourceGenerators. They allow to emit C# code to extend the existing code base. They will help us to analyze computed properties and help us to create their representation as an ExpressionTree.

Put all things together

Here is how we are gonna approach our goal:

Define attribute to mark computed properties that have to be understood by EF Core.
Define a structure of the EfFriendlyPropertiesLookupclass that will hold mapping between Computed Properties and respective Expression Tree.
Using SourceGenerators find all entities with computed properties marked with our attribute.
For each computed property, generate an Expression Tree and put it into EfFriendlyPropertiesLookup the class.
Write an interceptor that will consume the generated code and alter Expression trees in queries.
PROFIT

Create a generator

Create an attribute

In the beginning, we have to create an attribute to mark all computed properties we want to translate:

private const string EfFriendlyAttributeSource = @"
    namespace Lex45x.EntityFramework.ComputedProperties;
    /// <summary>
    /// Marks a property as one that has to be translated to respective ExpressionTree and substituted in EF queries
    /// </summary>
    [AttributeUsage(AttributeTargets.Property)]
    public class EfFriendlyAttribute : Attribute
    {
    }";

This attribute is defined as a constant string because our generator will later add it as a part of the source code of the project that references this package.

Here is how a generator will look like:

[Generator]
public class ComputedPropertiesGenerator : ISourceGenerator
{
    private const string EfFriendlyAttributeSource = @"
namespace Lex45x.EntityFramework.ComputedProperties;
/// <summary>
/// Marks a property as one that has to be translated to respective ExpressionTree and substituted in EF queries
/// </summary>
[AttributeUsage(AttributeTargets.Property)]
public class EfFriendlyAttribute : Attribute
{
}";

    public void Initialize(GeneratorInitializationContext context)
    {
       
    }

    public void Execute(GeneratorExecutionContext context)
    {
        context.AddSource("EfFriendlyAttribute.g.cs", SourceText.From(EfFriendlyAttributeSource, Encoding.UTF8));
    }
}

Now, our ComputedPropertiesGenerator will add our Attribute class as an EfFriendlyAttribute.g.cs file and include it in the compilation process of the referenced project.

Define the structure of the mapping class

For our purpose, we need a dictionary that will map respective PropertyInfo to Expression that implements the required behavior. Here how it might look like:

public static class EfFriendlyPropertiesLookup
{
    public static IReadOnlyDictionary<PropertyInfo, LambdaExpression> ComputedPropertiesExpression { get; } = new Dictionary<PropertyInfo, LambdaExpression>
    {
        [typeof(Employee).GetProperty("VacationBudget")] = (Expression<Func<Employee, double>>)((entity) => entity.VacationAllowance - entity.VacationRequests
            .Where(request => request.StartTime.Year == DateTime.UtcNow.Year)
            .Where(request => request.State == VacationRequestState.Approved || request.State == VacationRequestState.Pending)
            .Sum(request => request.TotalDays))
    };
}

A few things to note here:

The body of the expression tree is almost the same as the body of our computed property. The only difference is that instead of implicit this we have to use a parameter.
This implies that we can use only the public properties of the Employee class.

Now, let’s try to figure out, how we can generate a class similar to this one.

Find all computed properties

The next step would be to find all computed properties with the EfFriendly attribute. To achieve it, we will have to register our ISyntaxContextReceiver interface implementation.

In our case, we will have to filter properties marked with attributes and capture property implementation along with the property symbol (semantic representation of a class’ property). Here is what our syntax receiver would look like:

public class EntitySyntaxReceiver : ISyntaxContextReceiver
{
    public List<(IPropertySymbol Symbol, PropertyDeclarationSyntax Syntax)> Properties { get; } = new();

    public void OnVisitSyntaxNode(GeneratorSyntaxContext context)
    {
        // we care only about property declarations
        if (context.Node is not PropertyDeclarationSyntax { AttributeLists.Count: > 0 } propertyDeclarationSyntax)
        {
            return;
        }
        
        //we have to get a symbol to access information about attributes
        var declaredSymbol = (IPropertySymbol)context.SemanticModel.GetDeclaredSymbol(context.Node)!;
        var attributes = declaredSymbol.GetAttributes();

        if (!attributes.Any(data => data.AttributeClass?.ToDisplayString() == "EfFriendly"))
        {
            return;
        }

        Properties.Add((declaredSymbol, propertyDeclarationSyntax));
    }
}

Then we have to register it in our Generator’s Initialize method and use it in the Execute method:

[Generator]
public class ComputedPropertiesGenerator : ISourceGenerator
{
    private const string EfFriendlyAttributeSource = @"
namespace Lex45x.EntityFramework.ComputedProperties;
/// <summary>
/// Marks a property as one that has to be translated to respective ExpressionTree and substituted in EF queries
/// </summary>
[AttributeUsage(AttributeTargets.Property)]
public class EfFriendlyAttribute : Attribute
{
}";

    public void Initialize(GeneratorInitializationContext context)
    {
        //register our receiver
        context.RegisterForSyntaxNotifications(() => new EntitySyntaxReceiver());
    }

    public void Execute(GeneratorExecutionContext context)
    {
        //proceed only with EntitySyntaxReceiver in the context
        if (context.SyntaxContextReceiver is not EntitySyntaxReceiver receiver)
            return;

        context.AddSource("EfFriendlyAttribute.g.cs", SourceText.From(EfFriendlyAttributeSource, Encoding.UTF8));
}

Let’s see what kind of information we could capture with this approach.

IPropertySymbol — is similar to PropertyInfo as it contains information about property metadata. The key thing we care about is a property Name, property Type, and type that contains this property.
PropertyDeclarationSyntax — references the actual source code where the property is declared. For us, the most important part is that we can find the body of the Get-Method.

Here is the VacationBudget Get-Method body we can access:

"=> VacationAllowance - VacationRequests
        .Where(request => request.StartTime.Year == DateTime.UtcNow.Year)
        .Where(request => request.State == VacationRequestState.Approved || request.State == VacationRequestState.Pending)
        .Sum(request => request.TotalDays)"

Now, referring to a EfFriendlyPropertiesLookup class we want to create, we can see that it’s impossible to use the property body as-is. Because we have to replace all usages of implicit this with a usage of lambda parameter.

Fortunately, we can use CSharpSyntaxWalker it for this. It allows us to analyze components of a syntax node. So, knowing the type that contains the property, we can highlight all properties that were used in the Get-Method body:

public class ComputedPropertySymbolVisitor : CSharpSyntaxWalker
{
    private readonly INamedTypeSymbol currentType;
    public IReadOnlyList<string> UsedProperties => usedProperties;

    private readonly List<string> usedProperties = new();

    public ComputedPropertySymbolVisitor(INamedTypeSymbol currentType)
    {
        this.currentType = currentType;
    }

    public override void VisitIdentifierName(IdentifierNameSyntax node)
    {
        var referencedProperty = currentType.GetMembers(node.Identifier.ValueText);

        if (referencedProperty.Length > 0)
        {
            usedProperties.Add(node.Identifier.ValueText);
        }

        base.VisitIdentifierName(node);
    }
}

As we already need plenty of property information, so let’s create a class that would hold this information for us:

public class ComputedPropertyDeclaration
{
    public IPropertySymbol Symbol { get; }
    public PropertyDeclarationSyntax UnderlyingSyntax { get; }
    public IReadOnlyList<string> ReferencedProperties { get; }
    public ComputedPropertyDeclaration(IPropertySymbol symbol, PropertyDeclarationSyntax underlyingSyntax,
        IReadOnlyList<string> referencedProperties)
    {
        Symbol = symbol;
        UnderlyingSyntax = underlyingSyntax;
        ReferencedProperties = referencedProperties;
    }
}

And here is the final version of the EntitySyntaxReceiver:

public class EntitySyntaxReceiver : ISyntaxContextReceiver
{
    public List<ComputedPropertyDeclaration> Properties { get; } = new();

    public void OnVisitSyntaxNode(GeneratorSyntaxContext context)
    {
        if (context.Node is not PropertyDeclarationSyntax { AttributeLists.Count: > 0 } propertyDeclarationSyntax)
        {
            return;
        }

        var declaredSymbol = (IPropertySymbol)context.SemanticModel.GetDeclaredSymbol(context.Node)!;
        var attributes = declaredSymbol.GetAttributes();

        if (!attributes.Any(data => data.AttributeClass?.ToDisplayString() == "EfFriendly"))
        {
            return;
        }

        var visitor = new ComputedPropertySymbolVisitor(declaredSymbol.ContainingType);
        visitor.Visit(propertyDeclarationSyntax.ExpressionBody);

        Properties.Add(new ComputedPropertyDeclaration(declaredSymbol, propertyDeclarationSyntax,
            visitor.UsedProperties));
    }
}

Now, we have everything we need to generate our mapping class.

Mapping class generation

Let’s focus on the Generator's Execute method. We have to add source for a static class that will have a dictionary pre-initialized with all computed properties we’ve found. Here is what it will look like:

public void Execute(GeneratorExecutionContext context)
{
    if (context.SyntaxContextReceiver is not EntitySyntaxReceiver receiver)
        return;

    context.AddSource("EfFriendlyAttribute.g.cs", SourceText.From(EfFriendlyAttributeSource, Encoding.UTF8));
    
    var namespacesBuilder = new HashSet<string>();

    //making sure that all symbol namespaces will be imported
    foreach (var property in receiver.Properties)
    {
        namespacesBuilder.Add(property.Symbol.ContainingNamespace.ToString());
    }

    var computedPropertiesLookup = @$"
using System.Linq.Expressions;
using System.Reflection;
{namespacesBuilder.Aggregate(new StringBuilder(), (builder, s) => builder.AppendLine($"using {s};"))}

public static class EfFriendlyPropertiesLookup
{{
  public static IReadOnlyDictionary<PropertyInfo, Expression> ComputedPropertiesExpression {{ get; }} = new Dictionary<PropertyInfo, Expression>
  {{
    {receiver.Properties.Aggregate(new StringBuilder(), (builder, declaration) => builder.AppendLine($"[{declaration.GetPropertyInfoDeclaration()}] = {declaration.GetExpressionDeclaration()},"))}
  }};    
}}";
    context.AddSource("EfFriendlyPropertiesLookup.g.cs", SourceText.From(computedPropertiesLookup, Encoding.UTF8));
}

To clean our template I’ve delegated dictionary key and value generation to ComputedPropertyDeclaration class as it already has all of the necessary data in place. Here is the updated version of it:

public class ComputedPropertyDeclaration
{
    public IPropertySymbol Symbol { get; }
    public PropertyDeclarationSyntax UnderlyingSyntax { get; }
    public IReadOnlyList<string> ReferencedProperties { get; }

    public ComputedPropertyDeclaration(IPropertySymbol symbol, PropertyDeclarationSyntax underlyingSyntax,
        IReadOnlyList<string> referencedProperties)
    {
        Symbol = symbol;
        UnderlyingSyntax = underlyingSyntax;
        ReferencedProperties = referencedProperties;
    }

    public string GetExpressionDeclaration()
    {
        var getMethodBody = UnderlyingSyntax.ExpressionBody!.ToFullString();

        foreach (var usedProperty in ReferencedProperties)
        {
            getMethodBody = getMethodBody.Replace(usedProperty, $"entity.{usedProperty}");
        }

        return $"(Expression<Func<{Symbol.ContainingType.Name},{Symbol.Type.Name}>>) ((entity) {getMethodBody})";
    }

    public string GetPropertyInfoDeclaration()
    {
        return $"typeof({Symbol.ContainingType}).GetProperty(\"{Symbol.Name}\")";
    }
}

In the end, we will get a static class EfFriendlyPropertiesLookup built during compilation which contains all of the marked properties as well as their expression representation.

Now, we are ready to proceed with the EntityFramework part.

Writing Query Interceptor

We would have to create another project for the interceptor, as Source Generators should target netstandard2.0.

In this project, let’s create our implementation of IQueryExpressionInterceptor the interface:

public class ComputedPropertyCallInterceptor : IQueryExpressionInterceptor
{
    public Expression QueryCompilationStarting(Expression queryExpression, QueryExpressionEventData eventData)
    {
        throw new NotImplementedException();
    }
}

QueryCompilationStarting the method will be called just before our Expression tree is compiled to a database query. Let’s register our interceptor in the DbContext and see what kind of data we are getting there:

public class VacationSystemContext : DbContext
{
    public VacationSystemContext(DbContextOptions options) : base(options)
    {
    }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.AddInterceptors(new ComputedPropertyCallInterceptor());

        base.OnConfiguring(optionsBuilder);
    }

    public DbSet<Employee> Employees { get; private set; }
}

With the context being updated, let’s re-run our tests. As a reminder, here is a query we are executing in the test:

var query = dbContext.Employees.Select(employee => new
{
    employee.Id,
    employee.VacationBudget
}).ToQueryString();

And here is what we have in queryExpression parameter:

[Microsoft.EntityFrameworkCore.Query.EntityQueryRootExpression]
.Select(employee => new <>f__AnonymousType0`2(Id = employee.Id, VacationBudget = employee.VacationBudget))

Now, we can see that we are calling IQueryable.Select method and passing lambda as a parameter. Inside this lambda, we can see a call to an employee.VacationBudget.

The goal of our interceptor would be to change employee.VacationBudget to an expression from the generated EfFriendlyPropertiesLookup class.

entity => (Convert(entity.VacationAllowance, Double) 
  - entity.VacationRequests
    .Where(request => (request.StartTime.Year == DateTime.UtcNow.Year))
    .Where(request => ((Convert(request.State, Int32) == 1) OrElse (Convert(request.State, Int32) == 0))).Sum(request => request.TotalDays))

However, we can’t just take a lambda expression from the dictionary and use it, we will have to take a body of it and replace the entity parameter inside with a proper parameter reference from the queryExpression. Otherwise, our expression tree will be invalid.

Fortunately, dotnet has a special class that can solve both problems for us: ExpressionVisitor which enables us to traverse the expression tree and introduce changes to it. Let’s create a first visitor that will find all calls to computed properties:

internal class ComputedPropertiesVisitor : ExpressionVisitor
{
    private readonly IReadOnlyDictionary<PropertyInfo, LambdaExpression> configuration;

    public ComputedPropertiesVisitor(IReadOnlyDictionary<PropertyInfo, LambdaExpression> configuration)
    {
        this.configuration = configuration;
    }

    protected override Expression VisitMember(MemberExpression node)
    {
        if (node.Member is not PropertyInfo propertyInfo || !configuration.ContainsKey(propertyInfo))
        {
            return base.VisitMember(node);
        }

        var expression = configuration[propertyInfo];
        var resultExpression = expression.Body;
        var parametersToReplace = expression.Parameters;

        var parameterExpression = parametersToReplace[0];

        var expressionVisitor = new ReplaceParametersExpressionVisitor(parameterExpression, node.Expression);
        resultExpression = expressionVisitor.Visit(resultExpression);

        return resultExpression;
    }
}

ReplaceParametersExpressionVisitor is our second visitor that will take a body of expression and replace parameters in it:

internal class ReplaceParametersExpressionVisitor : ExpressionVisitor
{
    private readonly ParameterExpression propertyParameter;
    private readonly Expression expression;

    public ReplaceParametersExpressionVisitor(ParameterExpression propertyParameter, Expression expression)
    {
        this.propertyParameter = propertyParameter;
        this.expression = expression;
    }

    protected override Expression VisitParameter(ParameterExpression node)
    {
        return node == propertyParameter ? expression : base.VisitParameter(node);
    }
}

Now, let’s use those visitors inside an interceptor, and define a constructor parameter to take our mapping dictionary:

public class ComputedPropertyCallInterceptor : IQueryExpressionInterceptor
{
    private readonly ComputedPropertiesVisitor visitor;

    public ComputedPropertyCallInterceptor(IReadOnlyDictionary<PropertyInfo, LambdaExpression> configuration)
    {
        visitor = new ComputedPropertiesVisitor(configuration);
    }

    public Expression QueryCompilationStarting(Expression queryExpression, QueryExpressionEventData eventData)
    {
        return visitor.Visit(queryExpression);
    }
}

Consequently, we have to update our interceptor registration and use our EfFriendlyPropertiesLookup class:

public class VacationSystemContext : DbContext
{
    public VacationSystemContext(DbContextOptions options) : base(options)
    {
    }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.AddInterceptors(
            new ComputedPropertyCallInterceptor(EfFriendlyPropertiesLookup.ComputedPropertiesExpression));

        base.OnConfiguring(optionsBuilder);
    }

    public DbSet<Employee> Employees { get; private set; }
}

And that’s it! Let’s run our tests and compare the resulting queries.

Observe test execution changes

Here is a reminder of the test body:

public class SqlGenerationTests
{
    private VacationSystemContext dbContext = null!;

    [SetUp]
    public async Task Setup()
    {
        var dbContextOptionsBuilder = new DbContextOptionsBuilder<VacationSystemContext>().UseSqlite("DataSource=InMemory;Mode=Memory;Cache=Shared");
        dbContext = new VacationSystemContext(dbContextOptionsBuilder.Options);
        await dbContext.Database.OpenConnectionAsync();
        await dbContext.Database.EnsureDeletedAsync();
        var creationResult = await dbContext.Database.EnsureCreatedAsync();
    }

    [Test]
    public async Task VacationBudgetProjection()
    {
        var query = dbContext.Employees.Select(employee => new
        {
            employee.Id,
            employee.VacationBudget
        }).ToQueryString();
    }
}

And here is the definition of VacationBudget:

  //Vacation budget is a difference between Vacation Allowance and total days of pending and approved vacation requests
  [EfFriendly]
  public double VacationBudget => VacationAllowance - VacationRequests
      .Where(request => request.StartTime.Year == DateTime.UtcNow.Year)
      .Where(request => request.State == VacationRequestState.Approved || request.State == VacationRequestState.Pending)
      .Sum(request => request.TotalDays);

And here is what query we had without an interceptor:

SELECT "e"."Id"
FROM "Employees" AS "e"

Now, after all the work we’ve done, here is what query we are gonna have in the end:

SELECT "e"."Id", CAST("e"."VacationAllowance" AS REAL) - (
    SELECT COALESCE(SUM("v"."TotalDays"), 0.0)
    FROM "VacationRequest" AS "v"
    WHERE "e"."Id" = "v"."EmployeeId" AND CAST(strftime('%Y', "v"."StartTime") AS INTEGER) = CAST(strftime('%Y', 'now') AS INTEGER) AND "v"."State" IN (1, 0)) AS "VacationBudget"
FROM "Employees" AS "e"

It does exactly what we defined in the Computed Property body!

Limitations and known issues

This is not intended for production use!

A lot of things are not handled here, for example:

ExpressionTree incompatible property body will give a compilation error.
Multiple assemblies using the same analyzer are not tested and could give unpredicted behavior.
I’ve made no risk assessments for possible security issues.

However, if you find this thing helpful, feel free to contribute to my GitHub.

References

Repo with source code: https://github.com/Lex45x/Lex45x.EntityFramework.ComputedProperties

More about EF Interceptors:

More about expression trees:

More about Source Generators: