Add Exie AI assistant

docs/docs/self-hosting/kubernetes.md

@@ -53,6 +53,28 @@ The `provider` value determines what implementations to use for the various abst
 3. `EX_AppMode` should be set to `Production` if you want to send unrestricted emails.
 4. Please take a quick look at all the configuration options and settings that can be found in the various option classes located [here](https://github.com/exceptionless/Exceptionless/tree/master/src/Exceptionless.Core/Configuration).
 
+## Scaling Exie
+
+Exie does not require sticky sessions. The browser sends the retained visible conversation and current page context with every chat turn. Server-recorded tool results are retained briefly in the shared cache under the authenticated user, organization, and browser conversation id. Each new request can therefore be handled by any Exceptionless app replica without accepting tool output from the browser as trusted context. One streamed response remains connected to the replica that accepted it until that response completes. If that replica becomes unavailable, the user can retry the turn and the retry can be handled by another healthy replica.
+
+When running more than one app replica:
+
+1. Configure every replica with the same `EX_Assistant__*` values.
+2. Use shared Elasticsearch storage and the Redis-backed cache and message bus shown above. Redis makes organization usage limits atomic across replicas, and the shared message bus keeps access changes synchronized with browser connections.
+3. Configure the ingress or reverse proxy to stream responses without buffering and without a fixed request deadline. Keep an idle timeout of at least two minutes because an individual provider request can run for that long. The included Azure Application Gateway for Containers ingress configuration disables its request deadline and retains the gateway's idle timeout.
+
+Do not enable session affinity for Exie. The conversation contract uses the shared cache and is designed for replica failover.
+
+Assistant availability and customer usage allowances come from each organization's billing plan. Medium, Large, Extra Large, Enterprise, and hidden Unlimited plans allow 2/10, 3/15, 5/25, 10/50, and 20/100 concurrent turns/turns per minute respectively. Their calendar-month token and provider-cost safeguards are 25 million/$5, 50 million/$10, 100 million/$20, 250 million/$50, and 500 million/$100. Monthly and yearly variants have the same monthly allowance. Provider usage is accumulated for every model round in a turn, including tool-calling rounds, and the monthly token and cost limits are rechecked before each additional round.
+
+Exie usage is counted atomically in the shared cache and flushed into monthly organization records by the existing usage job. The records retain one year of accepted, completed, failed, and cancelled turns; provider requests; tool calls; prompt and completion tokens; provider cost; the plan id; and denials by concurrency, rate, token, or cost limit. Aggregate OpenTelemetry counters expose the same activity without organization-id labels, while the organization records identify which customers are consuming their allowance.
+
+Product-wide safety limits are intentionally not billing-plan options. Exie retains up to 20 visible messages and 48,000 visible-message characters, retains up to 48,000 characters of server-recorded tool context for 30 minutes, caps the complete provider input at 128,000 characters, and allows up to 2,048 output tokens, three tool rounds, and 12 tool calls per response. Tool calls may return at most 10 items, event details are capped at 16,384 characters, and an organization-wide turn may search at most five projects. A turn has a two-minute deadline. Provider routing is limited to models charging no more than $2 per million prompt tokens and $8 per million completion tokens.
+
+Use a dedicated provider API key for Exie and configure a provider-side monthly hard limit no higher than the amount you are willing to spend. The in-app controls depend on usage reported by the provider; the provider-side key limit is the final safeguard if usage reporting, Redis, or application configuration fails. Development mode uses the Unlimited plan's request allowances and bypasses plan access and organization usage limits.
+
+Setting `EX_Assistant__ApiKey` enables Exie by default. Set `EX_Assistant__Enabled=false` explicitly to keep the feature and its UI disabled even when an API key is configured.
+
 ## Active Directory Authentication
 
 To enable Active Directory authentication, update the Update the `exceptionless-config` config map to include the `EX_ConnectionStrings__LDAP` connection string. The value should be your domain's LDAP URI (e.g. `LDAP://ad.domain.com/` or `LDAP://ad.domain.com/DC=domain,DC=com`).

k8s/exceptionless/templates/app.yaml

@@ -155,6 +155,22 @@ spec:
         scheme: https
   {{- end }}
 
+---
+apiVersion: alb.networking.azure.io/v1
+kind: IngressExtension
+metadata:
+  name: {{ template "exceptionless.fullname" . }}-app-backend-settings
+spec:
+  backendSettings:
+    - service: {{ template "exceptionless.fullname" . }}-app
+      ports:
+        - port: 80
+          protocol: HTTP
+      timeouts:
+        # Exie streams multiple provider and tool rounds over one response. Disable the
+        # strict route deadline while the gateway's idle timeout still protects dead streams.
+        requestTimeout: 0s
+
 ---
 apiVersion: networking.k8s.io/v1
 kind: Ingress
@@ -164,6 +180,7 @@ metadata:
     alb.networking.azure.io/alb-name: {{ .Values.ingress.albName }}
     alb.networking.azure.io/alb-namespace: {{ .Values.ingress.albNamespace }}
     alb.networking.azure.io/alb-frontend: {{ template "exceptionless.fullname" . }}-fe
+    alb.networking.azure.io/alb-ingress-extension: {{ template "exceptionless.fullname" . }}-app-backend-settings
     cert-manager.io/cluster-issuer: {{ .Values.ingress.clusterIssuer }}
 spec:
   ingressClassName: azure-alb-external

src/Exceptionless.AppHost/Program.cs

@@ -1,11 +1,15 @@
 using System.Reflection;
+using Aspire.Hosting.ApplicationModel;
 using Aspire.Hosting.JavaScript;
 using Microsoft.Extensions.Hosting;
 
 string? scope = WorktreeScope.Resolve();
 bool isScoped = !String.IsNullOrWhiteSpace(scope);
 var worktreePorts = isScoped ? WorktreeScope.AssignFreePorts() : null;
 var builder = DistributedApplication.CreateBuilder(args);
+IResourceBuilder<ParameterResource>? assistantApiKey = !String.IsNullOrWhiteSpace(builder.Configuration["Parameters:assistant-api-key"])
+    ? builder.AddParameter("assistant-api-key", secret: true)
+    : null;
 bool servicesOnly = HasArgument("--services-only");
 bool ciE2E = HasArgument("--ci-e2e");
 bool includeDevTools = !ciE2E;
@@ -113,6 +117,11 @@
         .WithUrlForEndpoint("http", u => u.DisplayLocation = UrlDisplayLocation.DetailsOnly)
         .WithHttpHealthCheck("/health");
 
+    if (assistantApiKey is not null)
+    {
+        api.WithEnvironment("EX_Assistant__ApiKey", assistantApiKey);
+    }
+
     if (worktreePorts is not null)
     {
         api.WithEnvironment("Scope", scope!)

src/Exceptionless.Core/Billing/BillingPlans.cs

@@ -6,6 +6,42 @@ public class BillingPlans
 {
     public BillingPlans(AppOptions options)
     {
+        var mediumAssistantOptions = new AssistantPlanOptions
+        {
+            MaximumConcurrentTurns = 2,
+            MaximumTurnsPerMinute = 10,
+            MaximumMonthlyTokens = 25_000_000,
+            MaximumMonthlyCostUsd = 5m
+        };
+        var largeAssistantOptions = new AssistantPlanOptions
+        {
+            MaximumConcurrentTurns = 3,
+            MaximumTurnsPerMinute = 15,
+            MaximumMonthlyTokens = 50_000_000,
+            MaximumMonthlyCostUsd = 10m
+        };
+        var extraLargeAssistantOptions = new AssistantPlanOptions
+        {
+            MaximumConcurrentTurns = 5,
+            MaximumTurnsPerMinute = 25,
+            MaximumMonthlyTokens = 100_000_000,
+            MaximumMonthlyCostUsd = 20m
+        };
+        var enterpriseAssistantOptions = new AssistantPlanOptions
+        {
+            MaximumConcurrentTurns = 10,
+            MaximumTurnsPerMinute = 50,
+            MaximumMonthlyTokens = 250_000_000,
+            MaximumMonthlyCostUsd = 50m
+        };
+        var unlimitedAssistantOptions = new AssistantPlanOptions
+        {
+            MaximumConcurrentTurns = 20,
+            MaximumTurnsPerMinute = 100,
+            MaximumMonthlyTokens = 500_000_000,
+            MaximumMonthlyCostUsd = 100m
+        };
+
         FreePlan = new BillingPlan
         {
             Id = "EX_FREE",
@@ -55,7 +91,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = 25,
             RetentionDays = 90,
             MaxEventsPerMonth = 75000,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = mediumAssistantOptions
         };
 
         MediumYearlyPlan = new BillingPlan
@@ -68,7 +105,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = 25,
             RetentionDays = 90,
             MaxEventsPerMonth = 75000,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = mediumAssistantOptions
         };
 
         LargePlan = new BillingPlan
@@ -81,7 +119,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = -1,
             RetentionDays = 180,
             MaxEventsPerMonth = 250000,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = largeAssistantOptions
         };
 
         LargeYearlyPlan = new BillingPlan
@@ -94,7 +133,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = -1,
             RetentionDays = 180,
             MaxEventsPerMonth = 250000,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = largeAssistantOptions
         };
 
         ExtraLargePlan = new BillingPlan
@@ -107,7 +147,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = -1,
             RetentionDays = 180,
             MaxEventsPerMonth = 1000000,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = extraLargeAssistantOptions
         };
 
         ExtraLargeYearlyPlan = new BillingPlan
@@ -120,7 +161,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = -1,
             RetentionDays = 180,
             MaxEventsPerMonth = 1000000,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = extraLargeAssistantOptions
         };
 
         EnterprisePlan = new BillingPlan
@@ -133,7 +175,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = -1,
             RetentionDays = 180,
             MaxEventsPerMonth = 3000000,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = enterpriseAssistantOptions
         };
 
         EnterpriseYearlyPlan = new BillingPlan
@@ -146,7 +189,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = -1,
             RetentionDays = 180,
             MaxEventsPerMonth = 3000000,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = enterpriseAssistantOptions
         };
 
         UnlimitedPlan = new BillingPlan
@@ -160,7 +204,8 @@ public BillingPlans(AppOptions options)
             MaxUsers = -1,
             RetentionDays = options.MaximumRetentionDays,
             MaxEventsPerMonth = -1,
-            HasPremiumFeatures = true
+            HasPremiumFeatures = true,
+            Assistant = unlimitedAssistantOptions
         };
 
         Plans = new List<BillingPlan> { FreePlan, SmallYearlyPlan, MediumYearlyPlan, LargeYearlyPlan, ExtraLargeYearlyPlan, EnterpriseYearlyPlan, SmallPlan, MediumPlan, LargePlan, ExtraLargePlan, EnterprisePlan, UnlimitedPlan };
@@ -191,4 +236,8 @@ public BillingPlans(AppOptions options)
     public BillingPlan UnlimitedPlan { get; }
 
     public List<BillingPlan> Plans { get; }
+
+    public BillingPlan? GetPlan(string? planId) => planId is null
+        ? null
+        : Plans.FirstOrDefault(plan => String.Equals(plan.Id, planId, StringComparison.OrdinalIgnoreCase));
 }

src/Exceptionless.Core/Bootstrapper.cs

@@ -211,6 +211,7 @@ public static void RegisterServices(IServiceCollection services, AppOptions appO
         services.AddSingleton<SourceMapService>();
         services.AddSingleton<OAuthService>();
         services.AddSingleton<UsageService>();
+        services.AddSingleton<IAssistantUsageRecorder>(provider => provider.GetRequiredService<UsageService>());
         services.AddSingleton<SlackService>();
         services.AddSingleton<StackService>();
 

src/Exceptionless.Core/Configuration/AppOptions.cs

@@ -81,6 +81,7 @@ public class AppOptions
     public AuthOptions AuthOptions { get; internal set; } = null!;
     public OAuthServerOptions OAuthServerOptions { get; internal set; } = null!;
     public SourceMapOptions SourceMapOptions { get; internal set; } = null!;
+    public AssistantOptions AssistantOptions { get; internal set; } = null!;
 
     public static AppOptions ReadFromConfiguration(IConfiguration config)
     {
@@ -135,6 +136,7 @@ public static AppOptions ReadFromConfiguration(IConfiguration config)
         options.AuthOptions = AuthOptions.ReadFromConfiguration(config);
         options.OAuthServerOptions = OAuthServerOptions.ReadFromConfiguration(config);
         options.SourceMapOptions = SourceMapOptions.ReadFromConfiguration(config);
+        options.AssistantOptions = AssistantOptions.ReadFromConfiguration(config);
 
         return options;
     }

src/Exceptionless.Core/Configuration/AssistantOptions.cs

@@ -0,0 +1,26 @@
+using Microsoft.Extensions.Configuration;
+
+namespace Exceptionless.Core.Configuration;
+
+public sealed class AssistantOptions
+{
+    public bool Enabled { get; internal set; }
+    public bool IsConfigured => !String.IsNullOrWhiteSpace(ApiKey);
+    public bool IsAvailable => Enabled && IsConfigured;
+    public string? ApiKey { get; internal set; }
+    public string Endpoint { get; internal set; } = "https://openrouter.ai/api/v1/chat/completions";
+    public string Model { get; internal set; } = "deepseek/deepseek-v4-flash";
+
+    public static AssistantOptions ReadFromConfiguration(IConfiguration configuration)
+    {
+        var section = configuration.GetSection("Assistant");
+        string? apiKey = section.GetValue<string>(nameof(ApiKey));
+        return new AssistantOptions
+        {
+            Enabled = section.GetValue<bool?>(nameof(Enabled)) ?? !String.IsNullOrWhiteSpace(apiKey),
+            ApiKey = apiKey,
+            Endpoint = section.GetValue(nameof(Endpoint), "https://openrouter.ai/api/v1/chat/completions")!,
+            Model = section.GetValue(nameof(Model), "deepseek/deepseek-v4-flash")!
+        };
+    }
+}

src/Exceptionless.Core/Extensions/OrganizationExtensions.cs

@@ -112,6 +112,10 @@ public static void TrimUsage(this Organization organization, TimeProvider timePr
         var utcNow = timeProvider.GetUtcNow().UtcDateTime;
 
         // keep 1 year of usage
+        organization.AssistantUsage = organization.AssistantUsage.Except(organization.AssistantUsage
+            .Where(u => utcNow.Subtract(u.Date) > TimeSpan.FromDays(366)))
+            .ToList();
+
         organization.Usage = organization.Usage.Except(organization.Usage
             .Where(u => utcNow.Subtract(u.Date) > TimeSpan.FromDays(366)))
             .ToList();
@@ -122,6 +126,18 @@ public static void TrimUsage(this Organization organization, TimeProvider timePr
             .ToList();
     }
 
+    public static AssistantUsageInfo GetAssistantUsage(this Organization organization, DateTime date)
+    {
+        var startOfMonth = date.ToUniversalTime().StartOfMonth();
+        var usage = organization.AssistantUsage.FirstOrDefault(o => o.Date.Year == startOfMonth.Year && o.Date.Month == startOfMonth.Month);
+        if (usage is not null)
+            return usage;
+
+        usage = new AssistantUsageInfo { Date = startOfMonth };
+        organization.AssistantUsage.Add(usage);
+        return usage;
+    }
+
     public static UsageInfo GetCurrentUsage(this Organization organization, TimeProvider timeProvider)
     {
         return organization.GetUsage(timeProvider.GetUtcNow().UtcDateTime, timeProvider);

src/Exceptionless.Core/Models/AssistantUsageInfo.cs

@@ -0,0 +1,52 @@
+namespace Exceptionless.Core.Models;
+
+public record AssistantUsageInfo
+{
+    public DateTime Date { get; init; }
+    public string? PlanId { get; set; }
+    public long Turns { get; set; }
+    public long Completed { get; set; }
+    public long Failed { get; set; }
+    public long Cancelled { get; set; }
+    public long ProviderRequests { get; set; }
+    public long ToolCalls { get; set; }
+    public long PromptTokens { get; set; }
+    public long CompletionTokens { get; set; }
+    public long CostInMicrodollars { get; set; }
+    public long BlockedByConcurrency { get; set; }
+    public long BlockedByRateLimit { get; set; }
+    public long BlockedByTokenLimit { get; set; }
+    public long BlockedByCostLimit { get; set; }
+    public DateTime LastUsedUtc { get; set; }
+}
+
+public sealed record AssistantUsageIncrement
+{
+    public long Turns { get; init; }
+    public long Completed { get; init; }
+    public long Failed { get; init; }
+    public long Cancelled { get; init; }
+    public long ProviderRequests { get; init; }
+    public long ToolCalls { get; init; }
+    public long PromptTokens { get; init; }
+    public long CompletionTokens { get; init; }
+    public long CostInMicrodollars { get; init; }
+    public long BlockedByConcurrency { get; init; }
+    public long BlockedByRateLimit { get; init; }
+    public long BlockedByTokenLimit { get; init; }
+    public long BlockedByCostLimit { get; init; }
+
+    public bool HasValue => Turns > 0
+        || Completed > 0
+        || Failed > 0
+        || Cancelled > 0
+        || ProviderRequests > 0
+        || ToolCalls > 0
+        || PromptTokens > 0
+        || CompletionTokens > 0
+        || CostInMicrodollars > 0
+        || BlockedByConcurrency > 0
+        || BlockedByRateLimit > 0
+        || BlockedByTokenLimit > 0
+        || BlockedByCostLimit > 0;
+}

src/Exceptionless.Core/Models/Billing/BillingPlan.cs

@@ -1,4 +1,5 @@
 using System.Diagnostics;
+using System.Text.Json.Serialization;
 
 namespace Exceptionless.Core.Models.Billing;
 
@@ -15,4 +16,15 @@ public record BillingPlan
     public int MaxEventsPerMonth { get; init; }
     public bool HasPremiumFeatures { get; init; }
     public bool IsHidden { get; init; }
+
+    [JsonIgnore]
+    public AssistantPlanOptions? Assistant { get; init; }
+}
+
+public sealed record AssistantPlanOptions
+{
+    public int MaximumConcurrentTurns { get; init; }
+    public int MaximumTurnsPerMinute { get; init; }
+    public long MaximumMonthlyTokens { get; init; }
+    public decimal MaximumMonthlyCostUsd { get; init; }
 }