CMS/docs/monitoring-alerts-consolidated-report.md

# 📊 Monitoring & Alerts System - Consolidated Implementation Report

**Date**: 2025-11-30  
**Status**: ✅ Skeleton Implemented (30% Complete)  
**Build**: ✅ Success  

---

## 📋 Executive Summary

اسکلت کامل سیستم Monitoring & Alerts پیاده‌سازی شد. این سیستم شامل دو بخش اصلی است:
1. **Alert System**: اعلان‌های مدیریتی (Critical/Warning/Success) برای Admin
2. **User Notification System**: اعلان‌های کاربری (SMS/Email/Push) برای Users

فعلاً فقط Logging فعال است. Integration های اصلی (Sentry, Slack, SMS) آماده پیاده‌سازی هستند.

---

## 🏗️ Architecture Overview

```
┌─────────────────────────────────────────────────────────┐
│                  Application Layer                       │
│  ┌─────────────────────┐  ┌─────────────────────────┐  │
│  │  IAlertService      │  │ IUserNotificationService│  │
│  │  - Critical         │  │ - Commission Received   │  │
│  │  - Warning          │  │ - Club Activation       │  │
│  │  - Success          │  │ - Payout Error          │  │
│  └─────────────────────┘  └─────────────────────────┘  │
└─────────────────────────────────────────────────────────┘
                          ↓ implements
┌─────────────────────────────────────────────────────────┐
│              Infrastructure Layer                        │
│  ┌─────────────────────┐  ┌─────────────────────────┐  │
│  │   AlertService      │  │ UserNotificationService │  │
│  │   ✅ Logging        │  │   ✅ Logging            │  │
│  │   ⏳ Sentry         │  │   ⏳ SMS Gateway        │  │
│  │   ⏳ Slack          │  │   ⏳ Email Service      │  │
│  │   ⏳ Email          │  │   ⏳ Push Notification  │  │
│  └─────────────────────┘  └─────────────────────────┘  │
│                                                           │
│  ┌──────────────────────────────────────────────────┐   │
│  │         MonitoringSettings (Configuration)       │   │
│  │  - SentryEnabled, SentryDsn                      │   │
│  │  - SlackEnabled, SlackWebhookUrl                 │   │
│  │  - EmailAlertsEnabled, AdminEmails               │   │
│  │  - SmsNotificationsEnabled, SmsApiKey            │   │
│  └──────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────┘
                          ↓ used by
┌─────────────────────────────────────────────────────────┐
│              Background Workers / Handlers               │
│  ┌──────────────────────────────────────────────────┐   │
│  │   WeeklyNetworkCommissionWorker                  │   │
│  │   - On Success: SendSuccessNotificationAsync()   │   │
│  │   - On Error: SendCriticalAlertAsync()           │   │
│  └──────────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────────┐   │
│  │   ProcessUserPayoutsCommandHandler               │   │
│  │   - On Payout: SendCommissionReceivedNotification│   │
│  └──────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────┘
```

---

## 📦 Implementation Details

### 1️⃣ Alert Service (Admin Notifications)

**Interface**: `CMSMicroservice.Application/Common/Interfaces/IAlertService.cs`

```csharp
public interface IAlertService
{
    Task SendCriticalAlertAsync(string title, string message, Exception? exception, CancellationToken ct);
    Task SendWarningAlertAsync(string title, string message, CancellationToken ct);
    Task SendSuccessNotificationAsync(string title, string message, CancellationToken ct);
}
```

**Implementation**: `CMSMicroservice.Infrastructure/Services/Monitoring/AlertService.cs`

**Current Behavior**:
```
🚨 CRITICAL ALERT: {Title} - {Message}
⚠️ WARNING ALERT: {Title} - {Message}
✅ SUCCESS: {Title} - {Message}
```

**Pending Integrations**:
- **Sentry**: Exception tracking & aggregation (TODO: `SentrySdk.CaptureException()`)
- **Slack**: Real-time alerts to channel (TODO: HTTP POST to webhook)
- **Email**: Alert emails to admin list (TODO: SMTP integration)

---

### 2️⃣ User Notification Service

**Interface**: `CMSMicroservice.Application/Common/Interfaces/IAlertService.cs` (same file)

```csharp
public interface IUserNotificationService
{
    Task SendCommissionReceivedNotificationAsync(long userId, decimal amount, int weekNumber, CancellationToken ct);
    Task SendClubActivationNotificationAsync(long userId, CancellationToken ct);
    Task SendPayoutErrorNotificationAsync(long userId, string errorMessage, CancellationToken ct);
}
```

**Implementation**: `CMSMicroservice.Infrastructure/Services/Monitoring/UserNotificationService.cs`

**Current Behavior**:
```
📧 Sending commission notification: User={UserId}, Amount={Amount}, Week={WeekNumber}
🎉 Sending club activation notification: User={UserId}
⚠️ Sending payout error notification: User={UserId}, Error={Error}
```

**Pending Integrations**:
- **SMS Gateway**: Kavenegar/Ghasedak integration (TODO: HTTP API call)
- **Email Service**: SMTP/SendGrid integration (TODO: template-based emails)
- **Push Notification**: FCM/OneSignal integration (TODO: mobile app notifications)

---

### 3️⃣ Configuration Model

**File**: `CMSMicroservice.Infrastructure/Services/Monitoring/MonitoringSettings.cs`

```csharp
public class MonitoringSettings
{
    public const string SectionName = "Monitoring";
    
    // Sentry
    public bool SentryEnabled { get; set; }
    public string? SentryDsn { get; set; }
    
    // Slack
    public bool SlackEnabled { get; set; }
    public string? SlackWebhookUrl { get; set; }
    
    // Email Alerts (Admin)
    public bool EmailAlertsEnabled { get; set; }
    public List<string> AdminEmails { get; set; }
    
    // SMS (User Notifications)
    public bool SmsNotificationsEnabled { get; set; }
    public string? SmsApiKey { get; set; }
    public string? SmsGatewayUrl { get; set; }
}
```

**Config File**: `CMSMicroservice.WebApi/appsettings.json`

```json
{
  "Monitoring": {
    "SentryEnabled": false,
    "SentryDsn": "",
    "SlackEnabled": false,
    "SlackWebhookUrl": "",
    "EmailAlertsEnabled": false,
    "AdminEmails": ["admin@example.com"],
    "SmsNotificationsEnabled": false,
    "SmsApiKey": "",
    "SmsGatewayUrl": ""
  }
}
```

---

### 4️⃣ Dependency Injection

**File**: `CMSMicroservice.Infrastructure/ConfigureServices.cs`

```csharp
services.AddScoped<IAlertService, AlertService>();
services.AddScoped<IUserNotificationService, UserNotificationService>();
```

---

### 5️⃣ Worker Integration

**File**: `CMSMicroservice.Infrastructure/BackgroundJobs/WeeklyNetworkCommissionWorker.cs`

**On Success**:
```csharp
await alertService.SendSuccessNotificationAsync(
    "Weekly Commission Completed",
    $"Week {previousWeekNumber}: {payoutsProcessed} payouts, {balancesToExpire.Count} balances expired");
```

**On Error**:
```csharp
await alertService.SendCriticalAlertAsync(
    "Weekly Commission Worker Failed",
    $"Worker execution {executionId} failed for week {GetPreviousWeekNumber()}",
    ex,
    cancellationToken);
```

---

## 🔌 Integration Roadmap

### Priority 1: Sentry (High - 1 hour)

**Why**: Critical error tracking & aggregation برای Production

**Steps**:
1. Install NuGet:
   ```bash
   dotnet add package Sentry.AspNetCore
   ```

2. Configure in `Program.cs`:
   ```csharp
   builder.WebHost.UseSentry(options =>
   {
       options.Dsn = builder.Configuration["Monitoring:SentryDsn"];
       options.Environment = builder.Environment.EnvironmentName;
       options.TracesSampleRate = 1.0;
   });
   ```

3. Update `AlertService.SendCriticalAlertAsync()`:
   ```csharp
   if (_settings.SentryEnabled && exception != null)
   {
       SentrySdk.CaptureException(exception, scope =>
       {
           scope.SetTag("alert.title", title);
           scope.SetExtra("message", message);
       });
   }
   ```

4. Set DSN in `appsettings.Production.json`:
   ```json
   {
     "Monitoring": {
       "SentryEnabled": true,
       "SentryDsn": "https://xxxxx@sentry.io/12345"
     }
   }
   ```

---

### Priority 2: Slack Webhook (Medium - 2 hours)

**Why**: Real-time alerts به تیم Development/DevOps

**Steps**:
1. Create Incoming Webhook در Slack:
   - Go to: `https://api.slack.com/apps`
   - Create app → Incoming Webhooks → Add to channel
   - Copy Webhook URL

2. Update `AlertService`:
   ```csharp
   private readonly HttpClient _httpClient;
   
   public async Task SendCriticalAlertAsync(...)
   {
       _logger.LogCritical(exception, "🚨 {Title} - {Message}", title, message);
       
       if (_settings.SlackEnabled)
       {
           var payload = new
           {
               text = $"🚨 *{title}*",
               attachments = new[]
               {
                   new
                   {
                       color = "danger",
                       text = message,
                       fields = exception != null ? new[]
                       {
                           new { title = "Exception", value = exception.Message, @short = false }
                       } : null
                   }
               }
           };
           
           await _httpClient.PostAsJsonAsync(_settings.SlackWebhookUrl, payload);
       }
   }
   ```

3. Set Webhook URL in config:
   ```json
   {
     "Monitoring": {
       "SlackEnabled": true,
       "SlackWebhookUrl": "https://hooks.slack.com/services/T00/B00/XXX"
     }
   }
   ```

---

### Priority 3: SMS Gateway - Kavenegar (Medium - 3 hours)

**Why**: اطلاع‌رسانی کمیسیون به کاربران

**Steps**:
1. Get API Key from Kavenegar:
   - Sign up: `https://panel.kavenegar.com`
   - API Key: Settings → API Key

2. Create `ISmsGatewayService`:
   ```csharp
   public interface ISmsGatewayService
   {
       Task SendAsync(string mobile, string message, CancellationToken ct = default);
   }
   ```

3. Implement `KavenegarSmsService`:
   ```csharp
   public class KavenegarSmsService : ISmsGatewayService
   {
       private readonly HttpClient _httpClient;
       private readonly string _apiKey;
       
       public async Task SendAsync(string mobile, string message, CancellationToken ct)
       {
           var url = $"https://api.kavenegar.com/v1/{_apiKey}/sms/send.json";
           var payload = new
           {
               receptor = mobile,
               message = message
           };
           
           var response = await _httpClient.PostAsJsonAsync(url, payload, ct);
           response.EnsureSuccessStatusCode();
       }
   }
   ```

4. Update `UserNotificationService.SendCommissionReceivedNotificationAsync()`:
   ```csharp
   var user = await _context.Users.FindAsync(userId, ct);
   
   if (user.SmsNotifications && _settings.SmsNotificationsEnabled)
   {
       var message = $"کمیسیون شما: {amount:N0} ریال برای هفته {weekNumber} واریز شد.";
       await _smsGateway.SendAsync(user.Mobile, message, ct);
   }
   ```

5. Configure:
   ```json
   {
     "Monitoring": {
       "SmsNotificationsEnabled": true,
       "SmsApiKey": "your-kavenegar-api-key"
     }
   }
   ```

---

### Priority 4: Email Alerts for Admins (Low - 2 hours)

**Why**: Backup notification channel

**Options**:
- **A) MailKit (SMTP)**:
  ```csharp
  using var client = new SmtpClient();
  await client.ConnectAsync("smtp.gmail.com", 587, SecureSocketOptions.StartTls);
  await client.AuthenticateAsync("user@example.com", "password");
  
  var message = new MimeMessage();
  message.From.Add(new MailboxAddress("CMS Alerts", "noreply@foursat.ir"));
  message.To.Add(new MailboxAddress("Admin", adminEmail));
  message.Subject = $"[ALERT] {title}";
  message.Body = new TextPart("html") { Text = htmlMessage };
  
  await client.SendAsync(message);
  ```

- **B) SendGrid API**:
  ```csharp
  var client = new SendGridClient(_settings.SendGridApiKey);
  var msg = MailHelper.CreateSingleEmail(
      from: new EmailAddress("noreply@foursat.ir", "CMS Alerts"),
      to: new EmailAddress(adminEmail),
      subject: $"[ALERT] {title}",
      plainTextContent: message,
      htmlContent: htmlMessage
  );
  await client.SendEmailAsync(msg);
  ```

**Config**:
```json
{
  "Monitoring": {
    "EmailAlertsEnabled": true,
    "AdminEmails": ["admin@foursat.ir", "devops@foursat.ir"],
    "SmtpServer": "smtp.gmail.com",
    "SmtpPort": 587,
    "SmtpUsername": "user@example.com",
    "SmtpPassword": "password"
  }
}
```

---

### Priority 5: Retry Logic با Exponential Backoff (Low - 1 hour)

**Why**: بهبود Reliability در صورت خطاهای Transient

**Implementation در Worker**:
```csharp
private async Task<T> RetryWithExponentialBackoffAsync<T>(
    Func<Task<T>> operation,
    int maxRetries = 3,
    CancellationToken ct = default)
{
    for (int attempt = 0; attempt <= maxRetries; attempt++)
    {
        try
        {
            return await operation();
        }
        catch (Exception ex) when (attempt < maxRetries && IsTransientError(ex))
        {
            var delay = TimeSpan.FromSeconds(Math.Pow(2, attempt)); // 2^n: 1s, 2s, 4s
            
            _logger.LogWarning(ex,
                "Attempt {Attempt}/{MaxRetries} failed. Retrying in {Delay}s...",
                attempt + 1, maxRetries, delay.TotalSeconds);
            
            await Task.Delay(delay, ct);
        }
    }
    
    throw new InvalidOperationException($"Operation failed after {maxRetries} retries");
}

private bool IsTransientError(Exception ex)
{
    return ex is TimeoutException
        || ex is HttpRequestException
        || (ex is SqlException sqlEx && sqlEx.IsTransient);
}
```

**Usage**:
```csharp
// در ExecuteWeeklyCalculationAsync():
var balancesCalculated = await RetryWithExponentialBackoffAsync(async () =>
{
    return await mediator.Send(new CalculateWeeklyBalancesCommand
    {
        WeekNumber = previousWeekNumber
    }, cancellationToken);
}, maxRetries: 3, ct: cancellationToken);
```

---

## 🧪 Testing Guide

### Test 1: Alert Service (Console Logging)
```csharp
// در Controller یا Handler:
var alertService = _serviceProvider.GetRequiredService<IAlertService>();

await alertService.SendCriticalAlertAsync(
    "Test Critical Alert",
    "این یک تست برای Alert Service است",
    new Exception("Sample exception"));

await alertService.SendSuccessNotificationAsync(
    "Test Success",
    "عملیات با موفقیت انجام شد");
```

**Expected Output**:
```
🚨 CRITICAL ALERT: Test Critical Alert - این یک تست برای Alert Service است
✅ SUCCESS: Test Success - عملیات با موفقیت انجام شد
```

---

### Test 2: User Notification Service
```csharp
var notificationService = _serviceProvider.GetRequiredService<IUserNotificationService>();

await notificationService.SendCommissionReceivedNotificationAsync(
    userId: 123,
    amount: 500_000,
    weekNumber: 48);
```

**Expected Output**:
```
📧 Sending commission notification: User=123, Amount=500000, Week=48
```

---

### Test 3: Worker Integration
```bash
# Run Worker manually (for testing)
# تغییر زمان اجرا به 1 دقیقه بعد برای تست:
# در Worker: var delay = TimeSpan.FromMinutes(1);
dotnet run --project CMSMicroservice.WebApi
```

**Expected**:
- Worker starts
- After 1 minute → Executes calculation
- On success → Logs: `✅ SUCCESS: Weekly Commission Completed`
- On error → Logs: `🚨 CRITICAL ALERT: Weekly Commission Worker Failed`

---

### Test 4: Sentry Integration (بعد از پیاده‌سازی)
```csharp
// Throw یک exception برای تست:
throw new InvalidOperationException("Test Sentry integration");
```

**Check**: Sentry dashboard → Issues → باید exception جدید نمایش داده شود

---

### Test 5: Slack Integration (بعد از پیاده‌سازی)
```csharp
await alertService.SendCriticalAlertAsync("Test Slack", "Testing webhook integration", null);
```

**Check**: Slack channel → باید پیام جدید نمایش داده شود

---

### Test 6: SMS Integration (بعد از پیاده‌سازی)
```csharp
await notificationService.SendCommissionReceivedNotificationAsync(
    userId: YOUR_USER_ID,  // با شماره موبایل معتبر
    amount: 100_000,
    weekNumber: 48);
```

**Check**: موبایل کاربر → باید SMS دریافت شود

---

## 📊 Current Status & Progress

| Component | Status | Completion | Notes |
|-----------|--------|------------|-------|
| **Interfaces** | ✅ Done | 100% | `IAlertService`, `IUserNotificationService` |
| **Skeleton Implementations** | ✅ Done | 100% | Logging only |
| **Configuration Model** | ✅ Done | 100% | `MonitoringSettings` |
| **DI Registration** | ✅ Done | 100% | In `ConfigureServices.cs` |
| **Worker Integration** | ✅ Done | 100% | Success + Error alerts |
| **appsettings Structure** | ✅ Done | 100% | Monitoring section added |
| **Sentry Integration** | ⏳ Pending | 0% | Install package + configure DSN |
| **Slack Webhook** | ⏳ Pending | 0% | Create webhook + implement POST |
| **SMS Gateway** | ⏳ Pending | 0% | Choose provider + get API key |
| **Email Alerts** | ⏳ Pending | 0% | SMTP/SendGrid integration |
| **Retry Logic** | ⏳ Pending | 0% | Exponential backoff implementation |
| **Testing** | ⏳ Pending | 0% | Unit + Integration tests |

**Overall Progress**: 30% ✅ | 70% ⏳

---

## 📝 Important Notes

### 1. Production Readiness
- ⚠️ **فعلاً فقط Logging فعال است**
- ⚠️ برای Production **حداقل Sentry** باید فعال شود
- ⚠️ برای Critical systems حتماً Slack هم اضافه شود

### 2. User Preferences
- SMS/Email/Push باید بر اساس تنظیمات کاربر (`User.SmsNotifications`, etc.) ارسال شود
- در `UserNotificationService` باید ابتدا preferences چک شود

### 3. Rate Limiting
- برای SMS Gateway باید Rate Limiting در نظر گرفته شود
- پیشنهاد: استفاده از Queue (Hangfire/RabbitMQ) برای ارسال تعداد زیاد SMS

### 4. Cost Management
- SMS و Email هزینه دارند
- پیشنهاد: Batching برای ارسال گروهی
- پیشنهاد: Template-based messaging برای کاهش هزینه

### 5. Security
- API Keys در `appsettings.json` نباید commit شوند
- استفاده از Environment Variables یا Azure Key Vault
- مثال: `SmsApiKey: ${SMS_API_KEY}` در appsettings

### 6. Monitoring the Monitor
- خود Alert System هم باید Monitor شود
- اگر Slack/SMS fail شد، باید Fallback به Email یا Log باشد
- پیشنهاد: Dead Letter Queue برای failed notifications

---

## 🔗 File Reference Map

```
CMS/
├── src/
│   ├── CMSMicroservice.Application/
│   │   └── Common/
│   │       └── Interfaces/
│   │           └── IAlertService.cs ⭐
│   │
│   ├── CMSMicroservice.Infrastructure/
│   │   ├── Services/
│   │   │   └── Monitoring/
│   │   │       ├── AlertService.cs ⭐
│   │   │       ├── UserNotificationService.cs ⭐
│   │   │       └── MonitoringSettings.cs ⭐
│   │   │
│   │   ├── BackgroundJobs/
│   │   │   └── WeeklyNetworkCommissionWorker.cs ✏️ (Modified)
│   │   │
│   │   └── ConfigureServices.cs ✏️ (Modified)
│   │
│   └── CMSMicroservice.WebApi/
│       └── appsettings.json ✏️ (Modified)
│
└── docs/
    └── monitoring-alerts-implementation-report.md 📄 (This file)
```

**Legend**:
- ⭐ = New file created
- ✏️ = Existing file modified
- 📄 = Documentation

---

## 🚀 Next Action Items

### Immediate (این هفته):
1. ✅ Review this document
2. ⏳ Decision: کدام Integration اول؟ (پیشنهاد: Sentry)
3. ⏳ Get credentials:
   - Sentry DSN
   - Slack Webhook URL
   - SMS Gateway API Key

### Short-term (هفته آینده):
4. ⏳ Implement Sentry integration
5. ⏳ Implement Slack webhook
6. ⏳ Test in Staging environment

### Long-term (ماه آینده):
7. ⏳ Implement SMS Gateway (Kavenegar)
8. ⏳ Add Email alerts
9. ⏳ Implement Retry logic
10. ⏳ Write Unit/Integration tests
11. ⏳ Deploy to Production

---

## 📞 Contact & Support

**Implementation Questions**:
- Developer: GitHub Copilot (این گزارش)
- Review: Development Team

**Service Providers**:
- **Sentry**: https://sentry.io (Error tracking)
- **Slack**: https://api.slack.com/messaging/webhooks (Webhooks)
- **Kavenegar**: https://kavenegar.com (SMS Gateway - Iran)
- **Ghasedak**: https://ghasedak.me (SMS Gateway Alternative)
- **SendGrid**: https://sendgrid.com (Email service)

---

**Last Updated**: 2025-11-30  
**Build Status**: ✅ Success  
**Ready for**: Integration implementation

---

## 🎯 TL;DR (خلاصه برای رجوع سریع)

### چی ساخته شد:
- ✅ `IAlertService` + `AlertService` (Admin alerts)
- ✅ `IUserNotificationService` + `UserNotificationService` (User notifications)
- ✅ `MonitoringSettings` (Configuration model)
- ✅ Worker integration (Success/Error alerts)
- ✅ DI registration
- ✅ appsettings structure

### فعلاً چی کار می‌کنه:
- Logging به Console (🚨 Critical, ⚠️ Warning, ✅ Success)

### چی باید اضافه بشه:
1. **Sentry** - Error tracking (Priority: High)
2. **Slack** - Real-time alerts (Priority: Medium)
3. **SMS Gateway** - User notifications (Priority: Medium)
4. **Email** - Backup channel (Priority: Low)
5. **Retry Logic** - Reliability (Priority: Low)

### کجا باید نگاه کنی:
- Interfaces: `CMSMicroservice.Application/Common/Interfaces/IAlertService.cs`
- Implementations: `CMSMicroservice.Infrastructure/Services/Monitoring/`
- Worker: `CMSMicroservice.Infrastructure/BackgroundJobs/WeeklyNetworkCommissionWorker.cs`
- Config: `CMSMicroservice.WebApi/appsettings.json`

### چطوری تست کنی:
```csharp
await alertService.SendCriticalAlertAsync("Test", "Message", null);
// Output: 🚨 CRITICAL ALERT: Test - Message
```

### بعدش چیکار کنم:
1. Get Sentry DSN → Update appsettings.Production.json
2. Install `Sentry.AspNetCore` → Configure in Program.cs
3. Update `AlertService.SendCriticalAlertAsync()` → Add `SentrySdk.CaptureException()`
4. Test → Deploy