chylex/Discord-History-Tracker
1
0
Fork 0
mirror of https://github.com/chylex/Discord-History-Tracker.git synced 2025-08-17 10:31:41 +02:00
Code Activity

Compare commits

base: chylex:v36.2.beta
Branches Tags
chylex:master chylex:http-server chylex:master-browser-only chylex:wip-viewer-messages chylex:app-json-2 chylex:app-raw-messages
chylex:v47.1 chylex:v47.0 chylex:v46.0 chylex:v45.0 chylex:v44.0 chylex:v43.1 chylex:v43.0 chylex:v42.1 chylex:v42.0 chylex:v41.2 chylex:v41.1 chylex:v41.0 chylex:v40.0 chylex:v39.1 chylex:v39 chylex:v38 chylex:v37.2 chylex:v37.1 chylex:v37 chylex:v36.2.beta chylex:v36.1.beta chylex:v36.beta chylex:v35.3 chylex:v35.2 chylex:v35.1 chylex:v35 chylex:v34 chylex:v33.2 chylex:v33.1 chylex:v33 chylex:v32.1 chylex:v32 chylex:v31
..
compare: chylex:app-raw-messages
Branches Tags
chylex:master chylex:http-server chylex:master-browser-only chylex:wip-viewer-messages chylex:app-json-2 chylex:app-raw-messages
chylex:v47.1 chylex:v47.0 chylex:v46.0 chylex:v45.0 chylex:v44.0 chylex:v43.1 chylex:v43.0 chylex:v42.1 chylex:v42.0 chylex:v41.2 chylex:v41.1 chylex:v41.0 chylex:v40.0 chylex:v39.1 chylex:v39 chylex:v38 chylex:v37.2 chylex:v37.1 chylex:v37 chylex:v36.2.beta chylex:v36.1.beta chylex:v36.beta chylex:v35.3 chylex:v35.2 chylex:v35.1 chylex:v35 chylex:v34 chylex:v33.2 chylex:v33.1 chylex:v33 chylex:v32.1 chylex:v32 chylex:v31

1 Commits
v36.2.beta ... app-raw-me

Author SHA1 Message Date
chylex
dc5cd83da9 [WIP] Experimental Brotli compressing of raw message data 2022-03-05 15:59:57 +01:00
145 changed files with 6400 additions and 4167 deletions
Expand all files Collapse all files

1
.idea/Discord-History-Tracker.iml generated
View File

@@ -8,7 +8,6 @@
<excludeFolder url="file://$MODULE_DIR$/.vscode" />
<excludeFolder url="file://$MODULE_DIR$/bld" />
<excludeFolder url="file://$MODULE_DIR$/lib" />
<excludeFolder url="file://$MODULE_DIR$/app" />
</content>
<orderEntry type="jdk" jdkName="Python 3.8" jdkType="Python SDK" />
<orderEntry type="sourceFolder" forTests="false" />

52
.vscode/tasks.json vendored Normal file
View File

@@ -0,0 +1,52 @@
{
// See https://go.microsoft.com/fwlink/?LinkId=733558
// for the documentation about the tasks.json format
"version": "2.0.0",
"tasks": [
{
"label": "Build Raw",
"type": "shell",
"command": "python ./build.py --nominify --copytracker",
"group": {
"kind": "build",
"isDefault": true
},
"presentation": {
"reveal": "always",
"panel": "shared",
"echo": true,
"focus": false,
"showReuseMessage": false
},
"problemMatcher": []
},
{
"label": "Build Minified",
"type": "shell",
"command": "python ./build.py --copytracker",
"group": "build",
"presentation": {
"reveal": "always",
"panel": "shared",
"echo": true,
"focus": false,
"showReuseMessage": false
},
"problemMatcher": []
},
{
"label": "Build Website",
"type": "shell",
"command": "python ./build.py --website",
"group": "build",
"presentation": {
"reveal": "always",
"panel": "shared",
"echo": true,
"focus": false,
"showReuseMessage": false
},
"problemMatcher": []
}
]
}

59
README.md
View File

@@ -1,55 +1,48 @@
# Welcome
For instructions on how to **use Discord History Tracker**, visit the [official website](https://dht.chylex.com).
All you need to **use Discord History Tracker** is either an up-to-date browser, or the [Discord desktop client](https://discord.com/download). Visit the [official website](https://dht.chylex.com) for instructions.
To **report an issue or suggestion**, first please see the [issues](https://github.com/chylex/Discord-History-Tracker/issues) page and make sure someone else hasn't already created a similar issue report. If you do find an existing issue, comment on it or add a reaction. Otherwise, either click [New Issue](https://github.com/chylex/Discord-History-Tracker/issues/new), or contact me via email [contact@chylex.com](mailto:contact@chylex.com) or Twitter [@chylexmc](https://twitter.com/chylexmc).
If you are interested in **building from source code**, continue reading the [build instructions](#Build-Instructions) below.
This branch is dedicated to the Discord History Tracker desktop app. If you are looking for the older browser-only version, visit the [master-browser-only](https://github.com/chylex/Discord-History-Tracker/tree/master-browser-only) branch.
If you are interested in **creating your own version** from the source code, continue reading the [build instructions](#Build-Instructions) below.
# Build Instructions
Follow the steps below to create your own version of Discord History Tracker.
### Setup
Fork the repository and clone it to your computer (if you've never used git, you can download the [GitHub Desktop](https://desktop.github.com) client to get started quickly).
Folder organization:
* `app/` contains a Visual Studio solution for the desktop app
Now you can modify the source code:
* `src/tracker/` contains JS files that are automatically combined into the **tracker bookmark/script**
* `src/viewer/` contains HTML, CSS, JS files that are then combined into the **offline viewer page**
* `lib/` contains utilities required to build the project
* `web/` contains source code of the [official website](https://dht.chylex.com), which can be used as a template when making your own website
To start editing source code for the desktop app, install the [.NET 5 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/5.0), and then open `app/DiscordHistoryTracker.sln` in [Visual Studio](https://visualstudio.microsoft.com/downloads/) or [Rider](https://www.jetbrains.com/rider/).
### Building
To build a `Debug` version of the desktop app, there are no additional requirements.
After you've done changes to the source code, you will need to build it. Before that, download and install:
* (**required**) [Python 3](https://www.python.org/downloads)
* Use to run the build script
* (optional) [Node + npm](https://nodejs.org/en) & command line [uglify-js](https://www.npmjs.com/package/uglify-js)
* Not required on Windows
* Only required for optional [JS minification](#Minification) on Linux/Mac
To build a `Release` version of the desktop app, follow the instructions for your operating system.
Now open the folder that contains `build.py` in a command line, and run `python build.py` to create a build with default settings. The following files will be created:
* `bld/track.js` is the raw tracker script that can be pasted into a browser console
* `bld/track.html` is the tracker script but sanitized for inclusion in HTML (see `web/index.php` for examples)
* `bld/viewer.html` is the complete offline viewer
#### Release – Windows (64-bit)
You can tweak the build process using the following flags:
* `python build.py --nominify` to disable [minification](#Minification)
1. Install [Python 3](https://www.python.org/downloads), and ensure the `python` executable is in your `PATH`
2. Install [Powershell 5](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-on-windows) or newer (on Windows 10, the included version of Powershell should be enough)
### Minification
The `lib/` folder contains an installation of [Node](https://nodejs.org/en) and [uglify-js](https://www.npmjs.com/package/uglify-js), which are used to minify the tracking script. This installation will only work on 64-bit Windows; building on 32-bit Windows is not supported, but you can try.
The build process automatically minifies JS using `UglifyJS@3`, and CSS using a custom minifier.
Run the `app/build.bat` script, and read the [Distribution](#distribution) section below.
#### Release – Other Operating Systems
1. Install [Python 3](https://www.python.org/downloads), and ensure the `python` executable exists and launches Python 3
- On Debian and derivatives, you can install `python-is-python3`
- On other distributions, you can create a link manually, for ex. `ln -s /usr/bin/python3 /usr/bin/python`
- If you don't want `python` to mean Python 3, then edit `Desktop.csproj` and change `python` to `python3`
2. Install [Node + npm](https://nodejs.org/en)
3. Install [uglify-js](https://www.npmjs.com/package/uglify-js) globally (`npm install -g uglify-js`)
4. Install the `zip` package from your repository
Run the `app/build.sh` script, and read the [Distribution](#distribution) section below.
#### Distribution
The mentioned build scripts will prepare `Release` builds ready for distribution. Once the script finishes, the `app/bin` folder will contain self-contained executables for each major operating system, and a portable version that works on all other systems but requires .NET 5 to be installed.
Note that when building on Windows, the generated `.zip` files for Linux and Mac will not have correct file permissions, so it will not be possible to run them by double-clicking `DiscordHistoryTracker`. I tried using Python to re-create the archives with correct file permissions, but found that Linux `zip` tools could not see them. The only working solution is building the Windows + portable version on Windows, and Linux + Mac version on Linux.
* If the `--nominify` flag is used, minification will be completely disabled
* If `uglify-js` is not available from the command line, JS minification will be skipped
* When building on Windows 64-bit, the build script will use the included Node runner and packages
* When building on Windows 32-bit, you will need to download [Node 32-bit](https://nodejs.org/en/download) and replace the included one in `lib/`
* When building on Linux/Mac, the build script will attempt to find `uglifyjs` in the command line

12
app/.idea/.idea.DiscordHistoryTracker/.idea/avalonia.xml generated
View File

@@ -7,21 +7,15 @@
<entry key="Desktop/Dialogs/CheckBox/CheckBoxDialog.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Dialogs/Message/MessageDialog.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Dialogs/Progress/ProgressDialog.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Dialogs/TextBox/TextBoxDialog.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/AboutWindow.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Controls/AttachmentFilterPanel.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Controls/MessageFilterPanel.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Controls/ServerConfigurationPanel.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Controls/FilterPanel.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Controls/StatusBar.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/MainContentScreen.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/MainWindow.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Pages/AdvancedPage.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Pages/AttachmentsPage.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Pages/DatabasePage.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Pages/DebugPage.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Pages/TrackingPage.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Pages/ViewerPage.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Screens/MainContentScreen.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/Screens/WelcomeScreen.axaml" value="Desktop/Desktop.csproj" />
<entry key="Desktop/Main/WelcomeScreen.axaml" value="Desktop/Desktop.csproj" />
</map>
</option>
</component>

8
app/.idea/.idea.DiscordHistoryTracker/.idea/watcherTasks.xml generated Normal file
View File

@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectTasksOptions">
<enabled-global>
<option value="UglifyJS (Tracker)" />
</enabled-global>
</component>
</project>

151
app/Desktop/App.axaml
View File

@@ -1,56 +1,74 @@
<Application xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:common="clr-namespace:DHT.Desktop.Common"
xmlns:system="clr-namespace:System;assembly=System.Runtime"
x:Class="DHT.Desktop.App">
<Application.Styles>
<FluentTheme Mode="Light" />
<StyleInclude Source="avares://Avalonia.Controls.DataGrid/Themes/Default.xaml" />
<Style Selector="Button, CheckBox, RadioButton, Expander /template/ ToggleButton#ExpanderHeader">
<Setter Property="Cursor" Value="Hand" />
</Style>
<Style Selector="Button">
<Setter Property="Padding" Value="10 7 10 6" />
<Setter Property="Background" Value="#DDDDDD" />
<Setter Property="BorderBrush" Value="#999999" />
<Setter Property="BorderThickness" Value="1" />
<Setter Property="CornerRadius" Value="0" />
<Setter Property="HorizontalContentAlignment" Value="Center" />
<Setter Property="VerticalContentAlignment" Value="Center" />
</Style>
<Style Selector="Button /template/ ContentPresenter#PART_ContentPresenter">
<Setter Property="TextBlock.Foreground" Value="#060606" />
<Setter Property="Background" Value="#DDDDDD" />
<Setter Property="BorderBrush" Value="#999999" />
</Style>
<Style Selector="Button:pointerover /template/ ContentPresenter#PART_ContentPresenter">
<Setter Property="Background" Value="#E7E7E7" />
<Setter Property="BorderBrush" Value="#A2A2A2" />
<Setter Property="CornerRadius" Value="0" />
</Style>
<Style Selector="Button:pressed">
<Setter Property="RenderTransform" Value="none" />
</Style>
<Style Selector="Button:pressed /template/ ContentPresenter#PART_ContentPresenter">
<Setter Property="Background" Value="#EBEBEB" />
<Setter Property="BorderBrush" Value="#A5A5A5" />
<Setter Property="CornerRadius" Value="0" />
</Style>
<Style Selector="Button:disabled /template/ ContentPresenter#PART_ContentPresenter">
<Setter Property="TextBlock.Foreground" Value="#7A7A7A" />
<Setter Property="Background" Value="#E9E9E9" />
<Setter Property="BorderBrush" Value="#BFBFBF" />
</Style>
<Style Selector="TextBox">
<Setter Property="BorderBrush" Value="#999999" />
<Setter Property="BorderThickness" Value="1" />
<Setter Property="CornerRadius" Value="0" />
<Setter Property="SelectionBrush" Value="#72C0FF" />
<Setter Property="Background" Value="#F4F4F4" />
<Setter Property="Padding" Value="6 0" />
<Setter Property="VerticalContentAlignment" Value="Center" />
</Style>
<Style Selector="TextBox:pointerover /template/ Border#PART_BorderElement">
<Setter Property="BorderBrush" Value="#999999" />
<Setter Property="Background" Value="#F8F8F8" />
</Style>
<Style Selector="TextBox:focus /template/ Border#PART_BorderElement">
<Setter Property="BorderBrush" Value="#546A9F" />
<Setter Property="BorderThickness" Value="1" />
<Setter Property="Background" Value="#FBFBFB" />
</Style>
<Style Selector="TextBox:disabled /template/ Border#PART_BorderElement">
<Setter Property="BorderBrush" Value="#999999" />
<Setter Property="Background" Value="#BBBBBB" />
</Style>
<Style Selector="TextBox /template/ TextBlock#PART_Watermark">
<Setter Property="VerticalAlignment" Value="Center" />
</Style>
<Style Selector="TextBox:focus /template/ Border#PART_BorderElement">
<Setter Property="BorderThickness" Value="1" />
</Style>
<Style Selector="TextBox:disabled"><!-- TODO bug in Avalonia (https://github.com/AvaloniaUI/Avalonia/pull/7792) -->
<Setter Property="Foreground" Value="{DynamicResource TextControlForegroundDisabled}" />
</Style>
<Style Selector="TextBox:error DataValidationErrors">
<Style.Resources>
<ControlTemplate x:Key="InlineDataValidationContentTemplate" TargetType="DataValidationErrors">
<ContentPresenter Name="PART_ContentPresenter"
Padding="{TemplateBinding Padding}"
Background="{TemplateBinding Background}"
BorderThickness="{TemplateBinding BorderThickness}"
CornerRadius="{TemplateBinding CornerRadius}"
Content="{TemplateBinding Content}"
ContentTemplate="{TemplateBinding ContentTemplate}" />
</ControlTemplate>
</Style.Resources>
<Setter Property="Template" Value="{StaticResource InlineDataValidationContentTemplate}" />
</Style>
<Style Selector="Expander">
<Setter Property="CornerRadius" Value="0" />
</Style>
<Style Selector="Expander /template/ ToggleButton#ExpanderHeader">
<Setter Property="HorizontalContentAlignment" Value="Left" />
<Setter Property="Template">
@@ -90,93 +108,16 @@
</ControlTemplate>
</Setter>
</Style>
<Style Selector="Expander:expanded /template/ ToggleButton#ExpanderHeader /template/ Border#ExpandCollapseChevronBorder">
<Style.Animations>
<Animation FillMode="Both" Duration="0:0:0.0625">
<KeyFrame Cue="0%">
<Setter Property="RotateTransform.Angle" Value="180" />
</KeyFrame>
<KeyFrame Cue="100%">
<Setter Property="RotateTransform.Angle" Value="180" />
</KeyFrame>
</Animation>
</Style.Animations>
</Style>
<Style Selector="Expander:not(:expanded) /template/ ToggleButton#ExpanderHeader /template/ Border#ExpandCollapseChevronBorder">
<Style.Animations>
<Animation FillMode="Both" Duration="0:0:0.0625">
<KeyFrame Cue="0%">
<Setter Property="RotateTransform.Angle" Value="0" />
</KeyFrame>
<KeyFrame Cue="100%">
<Setter Property="RotateTransform.Angle" Value="0" />
</KeyFrame>
</Animation>
</Style.Animations>
</Style>
</Application.Styles>
<Application.Resources>
<common:NumberValueConverter x:Key="NumberValueConverter" />
<common:BytesValueConverter x:Key="BytesValueConverter" />
<system:Double x:Key="ControlContentThemeFontSize">14</system:Double>
<CornerRadius x:Key="ControlCornerRadius">0</CornerRadius>
<Color x:Key="SystemAccentColor">#3C5F95</Color>
<Color x:Key="SystemAccentColorDark1">#3C5F95</Color>
<Color x:Key="SystemAccentColorDark2">#3C5F95</Color>
<Color x:Key="SystemAccentColorDark3">#3C5F95</Color>
<Color x:Key="SystemAccentColorLight1">#3C5F95</Color>
<Color x:Key="SystemAccentColorLight2">#3C5F95</Color>
<Color x:Key="SystemAccentColorLight3">#3C5F95</Color>
<system:Double x:Key="ScrollBarSize">14</system:Double>
<TransformOperations x:Key="VerticalSmallScrollThumbScaleTransform">scaleX(0.5) translateX(-3px)</TransformOperations>
<TransformOperations x:Key="HorizontalSmallScrollThumbScaleTransform">scaleY(0.5) translateY(-3px)</TransformOperations>
<SolidColorBrush x:Key="ScrollBarPanningThumbBackground" Color="#8F8F8F" />
<SolidColorBrush x:Key="ButtonBorderBrush" Color="#3C5F95" />
<SolidColorBrush x:Key="ButtonBorderBrushPointerOver" Color="#0E2B59" />
<SolidColorBrush x:Key="ButtonBorderBrushPressed" Color="#061742" />
<SolidColorBrush x:Key="ButtonBorderBrushDisabled" Color="#9B9B9B" />
<SolidColorBrush x:Key="ButtonBackground" Color="#FFFFFF" />
<SolidColorBrush x:Key="ButtonBackgroundPointerOver" Color="#F6F9FD" />
<SolidColorBrush x:Key="ButtonBackgroundPressed" Color="#EDF3FD" />
<SolidColorBrush x:Key="ButtonBackgroundDisabled" Color="#FFFFFF" />
<SolidColorBrush x:Key="ButtonForeground" Color="#112961" />
<SolidColorBrush x:Key="ButtonForegroundPointerOver" Color="#050E41" />
<SolidColorBrush x:Key="ButtonForegroundPressed" Color="#010320" />
<SolidColorBrush x:Key="ButtonForegroundDisabled" Color="#8C8C8C" />
<SolidColorBrush x:Key="TextControlBorderBrush" Color="#515151" />
<SolidColorBrush x:Key="TextControlBorderBrushPointerOver" Color="#000000" />
<SolidColorBrush x:Key="TextControlBorderBrushFocused" Color="#3C5F95" />
<SolidColorBrush x:Key="TextControlBorderBrushDisabled" Color="#9B9B9B" />
<SolidColorBrush x:Key="TextControlBackground" Color="#FFFFFF" />
<SolidColorBrush x:Key="TextControlBackgroundPointerOver" Color="#FFFFFF" />
<SolidColorBrush x:Key="TextControlBackgroundFocused" Color="#F8FCFF" />
<SolidColorBrush x:Key="TextControlBackgroundDisabled" Color="#FFFFFF" />
<SolidColorBrush x:Key="TextControlForeground" Color="#000000" />
<SolidColorBrush x:Key="TextControlForegroundPointerOver" Color="#000000" />
<SolidColorBrush x:Key="TextControlForegroundFocused" Color="#000000" />
<SolidColorBrush x:Key="TextControlForegroundDisabled" Color="#8C8C8C" />
<SolidColorBrush x:Key="TextControlSelectionHighlightColor" Color="#DEE9F8" />
<SolidColorBrush x:Key="TextControlPlaceholderForeground" Color="#AAAAAA" />
<SolidColorBrush x:Key="TextControlPlaceholderForegroundPointerOver" Color="#AAAAAA" />
<SolidColorBrush x:Key="TextControlPlaceholderForegroundFocused" Color="#AAAAAA" />
<SolidColorBrush x:Key="TextControlPlaceholderForegroundDisabled" Color="#AAAAAA" />
<Thickness x:Key="ExpanderHeaderPadding">15,0</Thickness>
<Thickness x:Key="ExpanderContentPadding">12</Thickness>
<SolidColorBrush x:Key="ExpanderBorderBrush" Color="#697DAB" />
<SolidColorBrush x:Key="ExpanderBackground" Color="#697DAB" />
<SolidColorBrush x:Key="ExpanderForeground" Color="#FFFFFF" />
<SolidColorBrush x:Key="ExpanderChevronForeground" Color="#FFFFFF" />
<SolidColorBrush x:Key="ExpanderDropDownBorderBrush" Color="#697DAB" />
<SolidColorBrush x:Key="ExpanderDropDownBackground" Color="#FFFFFF" />
<Thickness x:Key="ExpanderContentPadding">15</Thickness>
<SolidColorBrush x:Key="ExpanderDropDownBackground" Color="#FCFCFC" />
</Application.Resources>

19
app/Desktop/Arguments.cs
View File

@@ -15,25 +15,12 @@ namespace DHT.Desktop {
for (int i = 0; i < args.Length; i++) {
string key = args[i];
switch (key) {
case "-debug":
Log.IsDebugEnabled = true;
continue;
}
string value;
if (i == 0 && !key.StartsWith('-')) {
value = key;
key = "-db";
}
else if (i >= args.Length - 1) {
if (i >= args.Length - 1) {
Log.Warn("Missing value for command line argument: " + key);
continue;
}
else {
value = args[++i];
}
string value = args[++i];
switch (key) {
case "-db":

45
app/Desktop/Common/BytesValueConverter.cs
View File

@@ -1,45 +0,0 @@
using System;
using System.Globalization;
using Avalonia.Data.Converters;
namespace DHT.Desktop.Common {
sealed class BytesValueConverter : IValueConverter {
private static readonly string[] Units = {
"B",
"kB",
"MB",
"GB",
"TB"
};
private const int Scale = 1000;
private static string Convert(ulong size) {
int power = size == 0L ? 0 : (int) Math.Log(size, Scale);
int unit = power >= Units.Length ? Units.Length - 1 : power;
if (unit == 0) {
return string.Format(Program.Culture, "{0:n0}", size) + " " + Units[unit];
}
else {
double humanReadableSize = size / Math.Pow(Scale, unit);
return string.Format(Program.Culture, "{0:n0}", humanReadableSize) + " " + Units[unit];
}
}
public object Convert(object? value, Type targetType, object? parameter, CultureInfo culture) {
if (value is long size and >= 0L) {
return Convert((ulong) size);
}
else if (value is ulong usize) {
return Convert(usize);
}
else {
return "-";
}
}
public object ConvertBack(object? value, Type targetType, object? parameter, CultureInfo culture) {
throw new NotSupportedException();
}
}
}

4
app/Desktop/Common/DatabaseGui.cs
View File

@@ -56,11 +56,11 @@ namespace DHT.Desktop.Common {
}
public static async Task<DialogResult.YesNo> ShowCanUpgradeDatabaseDialog(Window window) {
return await Dialog.ShowYesNo(window, "Database Upgrade", "This database was created with an older version of DHT. If you proceed, the database will be upgraded and will no longer open in previous versions of DHT.\n\nPlease ensure you have a backup of the database. Do you want to proceed with the upgrade?");
return await Dialog.ShowYesNo(window, "Database Upgrade", "This database was created with an older version of DHT. If you proceed, the database will be upgraded and will no longer open in previous versions of DHT. Do you want to proceed with the upgrade?");
}
public static async Task<DialogResult.YesNo> ShowCanUpgradeMultipleDatabaseDialog(Window window) {
return await Dialog.ShowYesNo(window, "Database Upgrade", "One or more databases were created with an older version of DHT. If you proceed, these databases will be upgraded and will no longer open in previous versions of DHT. Otherwise, these databases will be skipped.\n\nPlease ensure you have a backup of the databases. Do you want to proceed with the upgrade?");
return await Dialog.ShowYesNo(window, "Database Upgrade", "One or more databases were created with an older version of DHT. If you proceed, these databases will be upgraded and will no longer open in previous versions of DHT. Otherwise, these databases will be skipped. Do you want to proceed with the upgrade?");
}
}
}

2
app/Desktop/Common/NumberValueConverter.cs
View File

@@ -5,7 +5,7 @@ using Avalonia.Data.Converters;
namespace DHT.Desktop.Common {
sealed class NumberValueConverter : IValueConverter {
public object Convert(object? value, Type targetType, object? parameter, CultureInfo culture) {
return value == null ? "-" : string.Format(Program.Culture, "{0:n0}", value);
return string.Format(Program.Culture, "{0:n0}", value);
}
public object ConvertBack(object? value, Type targetType, object? parameter, CultureInfo culture) {

57
app/Desktop/Desktop.csproj
View File

@@ -3,7 +3,6 @@
<OutputType>WinExe</OutputType>
<TargetFramework>net5.0</TargetFramework>
<Nullable>enable</Nullable>
<AssemblyName>DiscordHistoryTracker</AssemblyName>
<RootNamespace>DHT.Desktop</RootNamespace>
<PackageId>DiscordHistoryTracker</PackageId>
<Authors>chylex</Authors>
@@ -12,6 +11,7 @@
<ApplicationIcon>./Resources/icon.ico</ApplicationIcon>
<CheckForOverflowUnderflow>true</CheckForOverflowUnderflow>
<SatelliteResourceLanguages>en</SatelliteResourceLanguages>
<AssemblyName>DiscordHistoryTracker</AssemblyName>
<GenerateAssemblyVersionAttribute>false</GenerateAssemblyVersionAttribute>
<GenerateAssemblyFileVersionAttribute>false</GenerateAssemblyFileVersionAttribute>
<GenerateAssemblyInformationalVersionAttribute>false</GenerateAssemblyInformationalVersionAttribute>
@@ -21,57 +21,56 @@
<DebugType>none</DebugType>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Avalonia" Version="0.10.14" />
<PackageReference Include="Avalonia.Controls.DataGrid" Version="0.10.14" />
<PackageReference Include="Avalonia.Desktop" Version="0.10.14" />
<PackageReference Include="Avalonia.Diagnostics" Version="0.10.14" Condition=" '$(Configuration)' == 'Debug' " />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Avalonia" Version="0.10.12" />
<PackageReference Include="Avalonia.Desktop" Version="0.10.12" />
<ProjectReference Include="..\Server\Server.csproj" />
</ItemGroup>
<ItemGroup Condition=" '$(Configuration)' == 'Debug' ">
<PackageReference Include="Avalonia.Diagnostics" Version="0.10.12" />
</ItemGroup>
<ItemGroup>
<Compile Include="..\Version.cs" Link="Version.cs" />
<Compile Update="Dialogs\TextBox\TextBoxDialog.axaml.cs">
</ItemGroup>
<ItemGroup>
<Compile Update="Windows\MainWindow.axaml.cs">
<DependentUpon>MainWindow.axaml</DependentUpon>
<SubType>Code</SubType>
</Compile>
<Compile Update="Dialogs\CheckBox\CheckBoxDialog.axaml.cs">
<DependentUpon>CheckBoxDialog.axaml</DependentUpon>
<SubType>Code</SubType>
</Compile>
<Compile Update="Dialogs\Progress\ProgressDialog.axaml.cs">
<DependentUpon>ProgressDialog.axaml</DependentUpon>
<SubType>Code</SubType>
</Compile>
<Compile Update="Dialogs\Message\MessageDialog.axaml.cs">
<DependentUpon>MessageDialog.axaml</DependentUpon>
<SubType>Code</SubType>
</Compile>
</ItemGroup>
<ItemGroup>
<UpToDateCheckInput Remove="Pages\DatabasePage.axaml" />
<UpToDateCheckInput Remove="Pages\TrackingPage.axaml" />
<UpToDateCheckInput Remove="Pages\ViewerPage.axaml" />
</ItemGroup>
<ItemGroup>
<AvaloniaResource Include="Resources/icon.ico" />
<EmbeddedResource Include="../Resources/Tracker/bootstrap.js">
<LogicalName>Tracker\%(RecursiveDir)%(Filename)%(Extension)</LogicalName>
<Link>Resources/Tracker/%(RecursiveDir)%(Filename)%(Extension)</Link>
<Visible>false</Visible>
</EmbeddedResource>
<EmbeddedResource Include="../Resources/Tracker/scripts/**" Condition=" '$(Configuration)' == 'Debug' ">
<EmbeddedResource Include="../Resources/Tracker/scripts.min/**">
<LogicalName>Tracker\scripts\%(RecursiveDir)%(Filename)%(Extension)</LogicalName>
<Link>Resources/Tracker/scripts/%(RecursiveDir)%(Filename)%(Extension)</Link>
<Visible>false</Visible>
</EmbeddedResource>
<EmbeddedResource Include="../Resources/Tracker/styles/**">
<LogicalName>Tracker\styles\%(RecursiveDir)%(Filename)%(Extension)</LogicalName>
<Link>Resources/Tracker/styles/%(RecursiveDir)%(Filename)%(Extension)</Link>
<Visible>false</Visible>
</EmbeddedResource>
<EmbeddedResource Include="../Resources/Viewer/**">
<LogicalName>Viewer\%(RecursiveDir)%(Filename)%(Extension)</LogicalName>
<Link>Resources/Viewer/%(RecursiveDir)%(Filename)%(Extension)</Link>
<Visible>false</Visible>
</EmbeddedResource>
<AvaloniaResource Include="Resources/icon.ico" />
</ItemGroup>
<Target Name="MinifyResources" BeforeTargets="PrepareForBuild" Condition=" '$(Configuration)' == 'Release' ">
<PropertyGroup>
<MinifiedResourceDir>$(ProjectDir)bin/.res/scripts</MinifiedResourceDir>
</PropertyGroup>
<ItemGroup>
<UpToDateCheckInput Include="$(ProjectDir)../Resources/Tracker/scripts/**" Visible="false" />
<EmbeddedResource Include="$(MinifiedResourceDir)/discord.js" LogicalName="Tracker\scripts\discord.js" Visible="false" />
<EmbeddedResource Include="$(MinifiedResourceDir)/dom.js" LogicalName="Tracker\scripts\dom.js" Visible="false" />
<EmbeddedResource Include="$(MinifiedResourceDir)/gui.js" LogicalName="Tracker\scripts\gui.js" Visible="false" />
<EmbeddedResource Include="$(MinifiedResourceDir)/settings.js" LogicalName="Tracker\scripts\settings.js" Visible="false" />
<EmbeddedResource Include="$(MinifiedResourceDir)/state.js" LogicalName="Tracker\scripts\state.js" Visible="false" />
</ItemGroup>
<RemoveDir Directories="$(ProjectDir)bin/.res/scripts" />
<Exec Command="python $(ProjectDir)../Resources/minify.py" WorkingDirectory="$(ProjectDir)../Resources" IgnoreExitCode="false" />
</Target>
</Project>

8
app/Desktop/Dialogs/CheckBox/CheckBoxDialog.axaml
View File

@@ -34,8 +34,8 @@
<StackPanel Margin="20">
<ScrollViewer MaxHeight="400">
<ItemsRepeater Items="{Binding Items}">
<ItemsRepeater.ItemTemplate>
<ItemsControl Items="{Binding Items}">
<ItemsControl.ItemTemplate>
<DataTemplate>
<CheckBox IsChecked="{Binding Checked}">
<Label>
@@ -43,8 +43,8 @@
</Label>
</CheckBox>
</DataTemplate>
</ItemsRepeater.ItemTemplate>
</ItemsRepeater>
</ItemsControl.ItemTemplate>
</ItemsControl>
</ScrollViewer>
<Panel Classes="buttons">
<WrapPanel Classes="left">

56
app/Desktop/Dialogs/TextBox/TextBoxDialog.axaml
View File

@@ -1,56 +0,0 @@
<Window xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:namespace="clr-namespace:DHT.Desktop.Dialogs.TextBox"
mc:Ignorable="d" d:DesignWidth="500"
x:Class="DHT.Desktop.Dialogs.TextBox.TextBoxDialog"
Title="{Binding Title}"
Icon="avares://DiscordHistoryTracker/Resources/icon.ico"
Width="500" SizeToContent="Height" CanResize="False"
WindowStartupLocation="CenterOwner">
<Window.DataContext>
<namespace:TextBoxDialogModel />
</Window.DataContext>
<Window.Styles>
<Style Selector="Panel.buttons">
<Setter Property="Margin" Value="0 20 0 0" />
</Style>
<Style Selector="Panel.buttons > WrapPanel.right">
<Setter Property="HorizontalAlignment" Value="Right" />
</Style>
<Style Selector="Panel.buttons Button">
<Setter Property="MinWidth" Value="80" />
<Setter Property="Margin" Value="8 0 0 0" />
</Style>
</Window.Styles>
<StackPanel Margin="20">
<ScrollViewer MaxHeight="400">
<StackPanel Spacing="10">
<TextBlock Text="{Binding Description}" TextWrapping="Wrap" />
<ItemsRepeater Items="{Binding Items}">
<ItemsRepeater.ItemTemplate>
<DataTemplate>
<DockPanel Margin="0 5 25 0">
<TextBox Name="Input" Text="{Binding Value}" Width="180" VerticalAlignment="Top" DockPanel.Dock="Right" />
<Label Target="Input" VerticalAlignment="Center" DockPanel.Dock="Left">
<TextBlock Text="{Binding Title}" TextWrapping="Wrap" />
</Label>
</DockPanel>
</DataTemplate>
</ItemsRepeater.ItemTemplate>
</ItemsRepeater>
</StackPanel>
</ScrollViewer>
<Panel Classes="buttons">
<WrapPanel Classes="right">
<Button Click="ClickOk" IsEnabled="{Binding !HasErrors}">OK</Button>
<Button Click="ClickCancel">Cancel</Button>
</WrapPanel>
</Panel>
</StackPanel>
</Window>

31
app/Desktop/Dialogs/TextBox/TextBoxDialog.axaml.cs
View File

@@ -1,31 +0,0 @@
using System.Diagnostics.CodeAnalysis;
using Avalonia;
using Avalonia.Controls;
using Avalonia.Interactivity;
using Avalonia.Markup.Xaml;
using DHT.Desktop.Dialogs.Message;
namespace DHT.Desktop.Dialogs.TextBox {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class TextBoxDialog : Window {
public TextBoxDialog() {
InitializeComponent();
#if DEBUG
this.AttachDevTools();
#endif
}
private void InitializeComponent() {
AvaloniaXamlLoader.Load(this);
}
public void ClickOk(object? sender, RoutedEventArgs e) {
Close(DialogResult.OkCancel.Ok);
}
public void ClickCancel(object? sender, RoutedEventArgs e) {
Close(DialogResult.OkCancel.Cancel);
}
}
}

47
app/Desktop/Dialogs/TextBox/TextBoxDialogModel.cs
View File

@@ -1,47 +0,0 @@
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using DHT.Utils.Models;
namespace DHT.Desktop.Dialogs.TextBox {
class TextBoxDialogModel : BaseModel {
public string Title { get; init; } = "";
public string Description { get; init; } = "";
private IReadOnlyList<TextBoxItem> items = Array.Empty<TextBoxItem>();
public IReadOnlyList<TextBoxItem> Items {
get => items;
protected set {
foreach (var item in items) {
item.ErrorsChanged -= OnItemErrorsChanged;
}
items = value;
foreach (var item in items) {
item.ErrorsChanged += OnItemErrorsChanged;
}
}
}
public bool HasErrors => Items.Any(static item => !item.IsValid);
private void OnItemErrorsChanged(object? sender, DataErrorsChangedEventArgs e) {
OnPropertyChanged(nameof(HasErrors));
}
}
sealed class TextBoxDialogModel<T> : TextBoxDialogModel {
public new IReadOnlyList<TextBoxItem<T>> Items { get; }
public IEnumerable<TextBoxItem<T>> ValidItems => Items.Where(static item => item.IsValid);
public TextBoxDialogModel(IEnumerable<TextBoxItem<T>> items) {
this.Items = new List<TextBoxItem<T>>(items);
base.Items = this.Items;
}
}
}

42
app/Desktop/Dialogs/TextBox/TextBoxItem.cs
View File

@@ -1,42 +0,0 @@
using System;
using System.Collections;
using System.ComponentModel;
using DHT.Utils.Models;
namespace DHT.Desktop.Dialogs.TextBox {
class TextBoxItem : BaseModel, INotifyDataErrorInfo {
public string Title { get; init; } = "";
public object? Item { get; init; } = null;
public Func<string, bool> ValidityCheck { get; init; } = static _ => true;
public bool IsValid => ValidityCheck(Value);
private string value = string.Empty;
public string Value {
get => this.value;
set {
Change(ref this.value, value);
ErrorsChanged?.Invoke(this, new DataErrorsChangedEventArgs(nameof(Value)));
}
}
public IEnumerable GetErrors(string? propertyName) {
if (propertyName == nameof(Value) && !IsValid) {
yield return string.Empty;
}
}
public bool HasErrors => !IsValid;
public event EventHandler<DataErrorsChangedEventArgs>? ErrorsChanged;
}
sealed class TextBoxItem<T> : TextBoxItem {
public new T Item { get; }
public TextBoxItem(T item) {
this.Item = item;
base.Item = item;
}
}
}

51
app/Desktop/Main/Controls/AttachmentFilterPanel.axaml
View File

@@ -1,51 +0,0 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:controls="clr-namespace:DHT.Desktop.Main.Controls"
mc:Ignorable="d"
x:Class="DHT.Desktop.Main.Controls.AttachmentFilterPanel">
<Design.DataContext>
<controls:AttachmentFilterPanelModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="WrapPanel > StackPanel">
<Setter Property="Margin" Value="0 20 40 0" />
<Setter Property="Spacing" Value="4" />
</Style>
<Style Selector="WrapPanel > StackPanel:nth-last-child(1)">
<Setter Property="Margin" Value="0 20 0 0" />
</Style>
<Style Selector="Grid > Label">
<Setter Property="VerticalAlignment" Value="Center" />
</Style>
<Style Selector="ComboBox">
<Setter Property="Margin" Value="8 0 0 0" />
</Style>
<Style Selector="Button">
<Setter Property="Margin" Value="0 0 0 8" />
</Style>
</UserControl.Styles>
<StackPanel>
<TextBlock Text="{Binding FilterStatisticsText}" />
<WrapPanel>
<StackPanel>
<CheckBox IsChecked="{Binding LimitSize}">Limit Size</CheckBox>
<StackPanel Orientation="Horizontal">
<TextBox Text="{Binding MaximumSize}" IsEnabled="{Binding LimitSize}" HorizontalContentAlignment="Right" />
<ComboBox IsEnabled="{Binding LimitSize}" Items="{Binding Units}" SelectedItem="{Binding MaximumSizeUnit}">
<ComboBox.ItemTemplate>
<DataTemplate>
<TextBlock Text="{Binding Name}" />
</DataTemplate>
</ComboBox.ItemTemplate>
</ComboBox>
</StackPanel>
</StackPanel>
</WrapPanel>
</StackPanel>
</UserControl>

17
app/Desktop/Main/Controls/AttachmentFilterPanel.axaml.cs
View File

@@ -1,17 +0,0 @@
using System.Diagnostics.CodeAnalysis;
using Avalonia.Controls;
using Avalonia.Markup.Xaml;
namespace DHT.Desktop.Main.Controls {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class AttachmentFilterPanel : UserControl {
public AttachmentFilterPanel() {
InitializeComponent();
}
private void InitializeComponent() {
AvaloniaXamlLoader.Load(this);
}
}
}

129
app/Desktop/Main/Controls/AttachmentFilterPanelModel.cs
View File

@@ -1,129 +0,0 @@
using System;
using System.Collections.Generic;
using System.ComponentModel;
using DHT.Desktop.Common;
using DHT.Server.Data.Filters;
using DHT.Server.Database;
using DHT.Utils.Models;
using DHT.Utils.Tasks;
namespace DHT.Desktop.Main.Controls {
sealed class AttachmentFilterPanelModel : BaseModel, IDisposable {
public sealed record Unit(string Name, uint Scale);
private static readonly Unit[] AllUnits = {
new ("B", 1),
new ("kB", 1024),
new ("MB", 1024 * 1024)
};
private static readonly HashSet<string> FilterProperties = new () {
nameof(LimitSize),
nameof(MaximumSize),
nameof(MaximumSizeUnit)
};
public string FilterStatisticsText { get; private set; } = "";
private bool limitSize = false;
private ulong maximumSize = 0L;
private Unit maximumSizeUnit = AllUnits[0];
public bool LimitSize {
get => limitSize;
set => Change(ref limitSize, value);
}
public ulong MaximumSize {
get => maximumSize;
set => Change(ref maximumSize, value);
}
public Unit MaximumSizeUnit {
get => maximumSizeUnit;
set => Change(ref maximumSizeUnit, value);
}
public IEnumerable<Unit> Units => AllUnits;
private readonly IDatabaseFile db;
private readonly string verb;
private readonly AsyncValueComputer<long> matchingAttachmentCountComputer;
private long? matchingAttachmentCount;
private long? totalAttachmentCount;
[Obsolete("Designer")]
public AttachmentFilterPanelModel() : this(DummyDatabaseFile.Instance) {}
public AttachmentFilterPanelModel(IDatabaseFile db, string verb = "Matches") {
this.db = db;
this.verb = verb;
this.matchingAttachmentCountComputer = AsyncValueComputer<long>.WithResultProcessor(SetAttachmentCounts).Build();
UpdateFilterStatistics();
PropertyChanged += OnPropertyChanged;
db.Statistics.PropertyChanged += OnDbStatisticsChanged;
}
public void Dispose() {
db.Statistics.PropertyChanged -= OnDbStatisticsChanged;
}
private void OnPropertyChanged(object? sender, PropertyChangedEventArgs e) {
if (e.PropertyName != null && FilterProperties.Contains(e.PropertyName)) {
UpdateFilterStatistics();
}
}
private void OnDbStatisticsChanged(object? sender, PropertyChangedEventArgs e) {
if (e.PropertyName == nameof(DatabaseStatistics.TotalAttachments)) {
totalAttachmentCount = db.Statistics.TotalAttachments;
UpdateFilterStatistics();
}
}
private void UpdateFilterStatistics() {
var filter = CreateFilter();
if (filter.IsEmpty) {
matchingAttachmentCountComputer.Cancel();
matchingAttachmentCount = totalAttachmentCount;
UpdateFilterStatisticsText();
}
else {
matchingAttachmentCount = null;
UpdateFilterStatisticsText();
matchingAttachmentCountComputer.Compute(() => db.CountAttachments(filter));
}
}
private void SetAttachmentCounts(long matchingAttachmentCount) {
this.matchingAttachmentCount = matchingAttachmentCount;
UpdateFilterStatisticsText();
}
private void UpdateFilterStatisticsText() {
var matchingAttachmentCountStr = matchingAttachmentCount?.Format() ?? "(...)";
var totalAttachmentCountStr = totalAttachmentCount?.Format() ?? "(...)";
FilterStatisticsText = verb + " " + matchingAttachmentCountStr + " out of " + totalAttachmentCountStr + " attachment" + (totalAttachmentCount is null or 1 ? "." : "s.");
OnPropertyChanged(nameof(FilterStatisticsText));
}
public AttachmentFilter CreateFilter() {
AttachmentFilter filter = new();
if (LimitSize) {
try {
filter.MaxBytes = maximumSize * maximumSizeUnit.Scale;
} catch (ArithmeticException) {
// set no size limit, because the overflown size is larger than any file could possibly be
}
}
return filter;
}
}
}

57
app/Desktop/Main/Controls/FilterPanel.axaml Normal file
View File

@@ -0,0 +1,57 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:controls="clr-namespace:DHT.Desktop.Main.Controls"
mc:Ignorable="d"
x:Class="DHT.Desktop.Main.Controls.FilterPanel">
<Design.DataContext>
<controls:FilterPanelModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="WrapPanel > StackPanel">
<Setter Property="Margin" Value="0 20 40 0" />
<Setter Property="Spacing" Value="4" />
</Style>
<Style Selector="Grid > Label">
<Setter Property="VerticalAlignment" Value="Center" />
</Style>
<Style Selector="Grid > CalendarDatePicker">
<Setter Property="CornerRadius" Value="0" />
<Setter Property="HorizontalAlignment" Value="Stretch" />
<Setter Property="HorizontalContentAlignment" Value="Center" />
<Setter Property="VerticalAlignment" Value="Center" />
<Setter Property="VerticalContentAlignment" Value="Center" />
<Setter Property="IsTodayHighlighted" Value="True" />
<Setter Property="SelectedDateFormat" Value="Short" />
</Style>
<Style Selector="Button">
<Setter Property="Margin" Value="0 0 0 8" />
</Style>
</UserControl.Styles>
<WrapPanel>
<StackPanel>
<CheckBox IsChecked="{Binding FilterByDate}">Filter by Date</CheckBox>
<Grid ColumnDefinitions="Auto, 4, 125" RowDefinitions="Auto, 4, Auto" Margin="4 0">
<Label Grid.Row="0" Grid.Column="0">From:</Label>
<CalendarDatePicker Grid.Row="0" Grid.Column="2" x:Name="StartDatePicker" IsEnabled="{Binding FilterByDate}" SelectedDateChanged="CalendarDatePicker_OnSelectedDateChanged" />
<Label Grid.Row="2" Grid.Column="0">To:</Label>
<CalendarDatePicker Grid.Row="2" Grid.Column="2" x:Name="EndDatePicker" IsEnabled="{Binding FilterByDate}" SelectedDateChanged="CalendarDatePicker_OnSelectedDateChanged" />
</Grid>
</StackPanel>
<StackPanel>
<CheckBox IsChecked="{Binding FilterByChannel}">Filter by Channel</CheckBox>
<Button Command="{Binding OpenChannelFilterDialog}" IsEnabled="{Binding FilterByChannel}">Select Channels...</Button>
<TextBlock Text="{Binding ChannelFilterLabel}" />
</StackPanel>
<StackPanel>
<CheckBox IsChecked="{Binding FilterByUser}">Filter by User</CheckBox>
<Button Command="{Binding OpenUserFilterDialog}" IsEnabled="{Binding FilterByUser}">Select Users...</Button>
<TextBlock Text="{Binding UserFilterLabel}" />
</StackPanel>
</WrapPanel>
</UserControl>

6
app/Desktop/Main/Controls/MessageFilterPanel.axaml.cs → app/Desktop/Main/Controls/FilterPanel.axaml.cs
View File

@@ -4,11 +4,11 @@ using Avalonia.Markup.Xaml;
namespace DHT.Desktop.Main.Controls {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class MessageFilterPanel : UserControl {
public sealed class FilterPanel : UserControl {
private CalendarDatePicker StartDatePicker => this.FindControl<CalendarDatePicker>("StartDatePicker");
private CalendarDatePicker EndDatePicker => this.FindControl<CalendarDatePicker>("EndDatePicker");
public MessageFilterPanel() {
public FilterPanel() {
InitializeComponent();
}
@@ -25,7 +25,7 @@ namespace DHT.Desktop.Main.Controls {
}
public void CalendarDatePicker_OnSelectedDateChanged(object? sender, SelectionChangedEventArgs e) {
if (DataContext is MessageFilterPanelModel model) {
if (DataContext is FilterPanelModel model) {
model.StartDate = StartDatePicker.SelectedDate;
model.EndDate = EndDatePicker.SelectedDate;
}

58
app/Desktop/Main/Controls/MessageFilterPanelModel.cs → app/Desktop/Main/Controls/FilterPanelModel.cs
View File

@@ -12,10 +12,9 @@ using DHT.Server.Data;
using DHT.Server.Data.Filters;
using DHT.Server.Database;
using DHT.Utils.Models;
using DHT.Utils.Tasks;
namespace DHT.Desktop.Main.Controls {
sealed class MessageFilterPanelModel : BaseModel, IDisposable {
sealed class FilterPanelModel : BaseModel {
private static readonly HashSet<string> FilterProperties = new () {
nameof(FilterByDate),
nameof(StartDate),
@@ -26,8 +25,6 @@ namespace DHT.Desktop.Main.Controls {
nameof(IncludedUsers)
};
public string FilterStatisticsText { get; private set; } = "";
public event PropertyChangedEventHandler? FilterPropertyChanged;
public bool HasAnyFilters => FilterByDate || FilterByChannel || FilterByUser;
@@ -91,23 +88,14 @@ namespace DHT.Desktop.Main.Controls {
private readonly Window window;
private readonly IDatabaseFile db;
private readonly string verb;
private readonly AsyncValueComputer<long> exportedMessageCountComputer;
private long? exportedMessageCount;
private long? totalMessageCount;
[Obsolete("Designer")]
public MessageFilterPanelModel() : this(null!, DummyDatabaseFile.Instance) {}
public FilterPanelModel() : this(null!, DummyDatabaseFile.Instance) {}
public MessageFilterPanelModel(Window window, IDatabaseFile db, string verb = "Matches") {
public FilterPanelModel(Window window, IDatabaseFile db) {
this.window = window;
this.db = db;
this.verb = verb;
this.exportedMessageCountComputer = AsyncValueComputer<long>.WithResultProcessor(SetExportedMessageCount).Build();
UpdateFilterStatistics();
UpdateChannelFilterLabel();
UpdateUserFilterLabel();
@@ -115,13 +103,8 @@ namespace DHT.Desktop.Main.Controls {
db.Statistics.PropertyChanged += OnDbStatisticsChanged;
}
public void Dispose() {
db.Statistics.PropertyChanged -= OnDbStatisticsChanged;
}
private void OnPropertyChanged(object? sender, PropertyChangedEventArgs e) {
if (e.PropertyName != null && FilterProperties.Contains(e.PropertyName)) {
UpdateFilterStatistics();
FilterPropertyChanged?.Invoke(sender, e);
}
@@ -134,11 +117,7 @@ namespace DHT.Desktop.Main.Controls {
}
private void OnDbStatisticsChanged(object? sender, PropertyChangedEventArgs e) {
if (e.PropertyName == nameof(DatabaseStatistics.TotalMessages)) {
totalMessageCount = db.Statistics.TotalMessages;
UpdateFilterStatistics();
}
else if (e.PropertyName == nameof(DatabaseStatistics.TotalChannels)) {
if (e.PropertyName == nameof(DatabaseStatistics.TotalChannels)) {
UpdateChannelFilterLabel();
}
else if (e.PropertyName == nameof(DatabaseStatistics.TotalUsers)) {
@@ -146,33 +125,6 @@ namespace DHT.Desktop.Main.Controls {
}
}
private void UpdateFilterStatistics() {
var filter = CreateFilter();
if (filter.IsEmpty) {
exportedMessageCountComputer.Cancel();
exportedMessageCount = totalMessageCount;
UpdateFilterStatisticsText();
}
else {
exportedMessageCount = null;
UpdateFilterStatisticsText();
exportedMessageCountComputer.Compute(() => db.CountMessages(filter));
}
}
private void SetExportedMessageCount(long exportedMessageCount) {
this.exportedMessageCount = exportedMessageCount;
UpdateFilterStatisticsText();
}
private void UpdateFilterStatisticsText() {
var exportedMessageCountStr = exportedMessageCount?.Format() ?? "(...)";
var totalMessageCountStr = totalMessageCount?.Format() ?? "(...)";
FilterStatisticsText = verb + " " + exportedMessageCountStr + " out of " + totalMessageCountStr + " message" + (totalMessageCount is null or 1 ? "." : "s.");
OnPropertyChanged(nameof(FilterStatisticsText));
}
public async void OpenChannelFilterDialog() {
var servers = db.GetAllServers().ToDictionary(static server => server.Id);
var items = new List<CheckBoxItem<ulong>>();
@@ -255,7 +207,7 @@ namespace DHT.Desktop.Main.Controls {
if (FilterByDate) {
filter.StartDate = StartDate;
filter.EndDate = EndDate?.AddDays(1).AddMilliseconds(-1);
filter.EndDate = EndDate;
}
if (FilterByChannel) {

63
app/Desktop/Main/Controls/MessageFilterPanel.axaml
View File

@@ -1,63 +0,0 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:controls="clr-namespace:DHT.Desktop.Main.Controls"
mc:Ignorable="d"
x:Class="DHT.Desktop.Main.Controls.MessageFilterPanel">
<Design.DataContext>
<controls:MessageFilterPanelModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="WrapPanel > StackPanel">
<Setter Property="Margin" Value="0 20 40 0" />
<Setter Property="Spacing" Value="4" />
</Style>
<Style Selector="WrapPanel > StackPanel:nth-last-child(1)">
<Setter Property="Margin" Value="0 20 0 0" />
</Style>
<Style Selector="Grid > Label">
<Setter Property="VerticalAlignment" Value="Center" />
</Style>
<Style Selector="Grid > CalendarDatePicker">
<Setter Property="CornerRadius" Value="0" />
<Setter Property="HorizontalAlignment" Value="Stretch" />
<Setter Property="HorizontalContentAlignment" Value="Center" />
<Setter Property="VerticalAlignment" Value="Center" />
<Setter Property="VerticalContentAlignment" Value="Center" />
<Setter Property="IsTodayHighlighted" Value="True" />
<Setter Property="SelectedDateFormat" Value="Short" />
</Style>
<Style Selector="Button">
<Setter Property="Margin" Value="0 0 0 8" />
</Style>
</UserControl.Styles>
<StackPanel>
<TextBlock Text="{Binding FilterStatisticsText}" />
<WrapPanel>
<StackPanel>
<CheckBox IsChecked="{Binding FilterByDate}">Filter by Date</CheckBox>
<Grid ColumnDefinitions="Auto, 4, 125" RowDefinitions="Auto, 4, Auto" Margin="4 0">
<Label Grid.Row="0" Grid.Column="0">From:</Label>
<CalendarDatePicker Grid.Row="0" Grid.Column="2" x:Name="StartDatePicker" IsEnabled="{Binding FilterByDate}" SelectedDateChanged="CalendarDatePicker_OnSelectedDateChanged" />
<Label Grid.Row="2" Grid.Column="0">To:</Label>
<CalendarDatePicker Grid.Row="2" Grid.Column="2" x:Name="EndDatePicker" IsEnabled="{Binding FilterByDate}" SelectedDateChanged="CalendarDatePicker_OnSelectedDateChanged" />
</Grid>
</StackPanel>
<StackPanel>
<CheckBox IsChecked="{Binding FilterByChannel}">Filter by Channel</CheckBox>
<Button Command="{Binding OpenChannelFilterDialog}" IsEnabled="{Binding FilterByChannel}">Select Channels...</Button>
<TextBlock Text="{Binding ChannelFilterLabel}" />
</StackPanel>
<StackPanel>
<CheckBox IsChecked="{Binding FilterByUser}">Filter by User</CheckBox>
<Button Command="{Binding OpenUserFilterDialog}" IsEnabled="{Binding FilterByUser}">Select Users...</Button>
<TextBlock Text="{Binding UserFilterLabel}" />
</StackPanel>
</WrapPanel>
</StackPanel>
</UserControl>

48
app/Desktop/Main/Controls/ServerConfigurationPanel.axaml
View File

@@ -1,48 +0,0 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:controls="clr-namespace:DHT.Desktop.Main.Controls"
mc:Ignorable="d"
x:Class="DHT.Desktop.Main.Controls.ServerConfigurationPanel">
<Design.DataContext>
<controls:ServerConfigurationPanelModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="TextBox">
<Setter Property="FontFamily" Value="Consolas,Courier" />
<Setter Property="FontSize" Value="15" />
</Style>
<Style Selector="WrapPanel > StackPanel">
<Setter Property="Orientation" Value="Vertical" />
<Setter Property="Margin" Value="0 0 10 10" />
</Style>
</UserControl.Styles>
<StackPanel>
<Button Command="{Binding OnClickToggleServerButton}" Content="{Binding ToggleServerButtonText}" IsEnabled="{Binding IsToggleServerButtonEnabled}" />
<TextBlock TextWrapping="Wrap" Margin="0 15">
The following settings determine how the tracking script communicates with this application. If you change them, you will have to copy/paste the tracking script again.
</TextBlock>
<WrapPanel>
<StackPanel>
<Label Target="Port">Port</Label>
<TextBox x:Name="Port" Width="70" Text="{Binding InputPort}" />
</StackPanel>
<StackPanel>
<Label Target="Token">Token</Label>
<TextBox x:Name="Token" Width="200" Text="{Binding InputToken}" />
</StackPanel>
<StackPanel VerticalAlignment="Bottom">
<Button Command="{Binding OnClickRandomizeToken}">Randomize Token</Button>
</StackPanel>
</WrapPanel>
<StackPanel Orientation="Horizontal" Spacing="10">
<Button IsEnabled="{Binding HasMadeChanges}" Command="{Binding OnClickApplyChanges}">Apply &amp; Restart</Button>
<Button IsEnabled="{Binding HasMadeChanges}" Command="{Binding OnClickCancelChanges}">Cancel</Button>
</StackPanel>
</StackPanel>
</UserControl>

16
app/Desktop/Main/Controls/ServerConfigurationPanel.axaml.cs
View File

@@ -1,16 +0,0 @@
using System.Diagnostics.CodeAnalysis;
using Avalonia.Controls;
using Avalonia.Markup.Xaml;
namespace DHT.Desktop.Main.Controls {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class ServerConfigurationPanel : UserControl {
public ServerConfigurationPanel() {
InitializeComponent();
}
private void InitializeComponent() {
AvaloniaXamlLoader.Load(this);
}
}
}

116
app/Desktop/Main/Controls/ServerConfigurationPanelModel.cs
View File

@@ -1,116 +0,0 @@
using System;
using Avalonia.Controls;
using DHT.Desktop.Dialogs.Message;
using DHT.Desktop.Server;
using DHT.Server.Database;
using DHT.Server.Service;
using DHT.Utils.Models;
namespace DHT.Desktop.Main.Controls {
sealed class ServerConfigurationPanelModel : BaseModel, IDisposable {
private string inputPort;
public string InputPort {
get => inputPort;
set {
Change(ref inputPort, value);
OnPropertyChanged(nameof(HasMadeChanges));
}
}
private string inputToken;
public string InputToken {
get => inputToken;
set {
Change(ref inputToken, value);
OnPropertyChanged(nameof(HasMadeChanges));
}
}
public bool HasMadeChanges => ServerManager.Port.ToString() != InputPort || ServerManager.Token != InputToken;
private bool isToggleServerButtonEnabled = true;
public bool IsToggleServerButtonEnabled {
get => isToggleServerButtonEnabled;
set => Change(ref isToggleServerButtonEnabled, value);
}
public string ToggleServerButtonText => serverManager.IsRunning ? "Stop Server" : "Start Server";
public event EventHandler<StatusBarModel.Status>? ServerStatusChanged;
private readonly Window window;
private readonly ServerManager serverManager;
[Obsolete("Designer")]
public ServerConfigurationPanelModel() : this(null!, new ServerManager(DummyDatabaseFile.Instance)) {}
public ServerConfigurationPanelModel(Window window, ServerManager serverManager) {
this.window = window;
this.serverManager = serverManager;
this.inputPort = ServerManager.Port.ToString();
this.inputToken = ServerManager.Token;
}
public void Initialize() {
ServerLauncher.ServerStatusChanged += ServerLauncherOnServerStatusChanged;
}
public void Dispose() {
ServerLauncher.ServerStatusChanged -= ServerLauncherOnServerStatusChanged;
}
private void ServerLauncherOnServerStatusChanged(object? sender, EventArgs e) {
ServerStatusChanged?.Invoke(this, serverManager.IsRunning ? StatusBarModel.Status.Ready : StatusBarModel.Status.Stopped);
OnPropertyChanged(nameof(ToggleServerButtonText));
IsToggleServerButtonEnabled = true;
}
private void BeforeServerStart() {
IsToggleServerButtonEnabled = false;
ServerStatusChanged?.Invoke(this, StatusBarModel.Status.Starting);
}
private void StartServer() {
BeforeServerStart();
serverManager.Launch();
}
private void StopServer() {
IsToggleServerButtonEnabled = false;
ServerStatusChanged?.Invoke(this, StatusBarModel.Status.Stopping);
serverManager.Stop();
}
public void OnClickToggleServerButton() {
if (serverManager.IsRunning) {
StopServer();
}
else {
StartServer();
}
}
public void OnClickRandomizeToken() {
InputToken = ServerUtils.GenerateRandomToken(20);
}
public async void OnClickApplyChanges() {
if (!ushort.TryParse(InputPort, out ushort port)) {
await Dialog.ShowOk(window, "Invalid Port", "Port must be a number between 0 and 65535.");
return;
}
BeforeServerStart();
serverManager.Relaunch(port, InputToken);
OnPropertyChanged(nameof(HasMadeChanges));
}
public void OnClickCancelChanges() {
InputPort = ServerManager.Port.ToString();
InputToken = ServerManager.Token;
}
}
}

15
app/Desktop/Main/Controls/StatusBar.axaml
View File

@@ -11,35 +11,32 @@
</Design.DataContext>
<UserControl.Background>
<SolidColorBrush>#3C4F79</SolidColorBrush>
<SolidColorBrush>#546A9F</SolidColorBrush>
</UserControl.Background>
<UserControl.Styles>
<Style Selector="StackPanel > TextBlock">
<Setter Property="Foreground" Value="#E7E7E7" />
<Setter Property="Foreground" Value="#E0E0E0" />
</Style>
<Style Selector="StackPanel > TextBlock.label">
<Setter Property="FontSize" Value="15" />
<Setter Property="FontWeight" Value="SemiLight" />
</Style>
<Style Selector="StackPanel > TextBlock.value">
<Setter Property="FontSize" Value="16" />
<Setter Property="FontWeight" Value="SemiLight" />
<Setter Property="TextAlignment" Value="Right" />
<Setter Property="Margin" Value="0 1" />
</Style>
<Style Selector="StackPanel > Rectangle">
<Setter Property="Margin" Value="14 1" />
<Setter Property="Stroke" Value="#697899" />
<Setter Property="Margin" Value="14 0" />
<Setter Property="Stroke" Value="#3B5287" />
<Setter Property="StrokeThickness" Value="2" />
<Setter Property="VerticalAlignment" Value="Stretch" />
</Style>
</UserControl.Styles>
<StackPanel Orientation="Horizontal" Margin="6 3">
<StackPanel Orientation="Horizontal" Margin="4 3">
<StackPanel Orientation="Vertical" Width="65">
<TextBlock Classes="label">Status</TextBlock>
<TextBlock FontSize="12" Margin="0 3 0 0" Text="{Binding StatusText}" />
<TextBlock FontSize="12" Margin="0 2 0 0" Text="{Binding StatusText}" />
</StackPanel>
<Rectangle />
<StackPanel Orientation="Vertical">

91
app/Desktop/Main/MainContentScreen.axaml Normal file
View File

@@ -0,0 +1,91 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:main="clr-namespace:DHT.Desktop.Main"
xmlns:controls="clr-namespace:DHT.Desktop.Main.Controls"
mc:Ignorable="d" d:DesignWidth="800" d:DesignHeight="450"
x:Class="DHT.Desktop.Main.MainContentScreen">
<Design.DataContext>
<main:MainContentScreenModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="TabControl ItemsPresenter > WrapPanel">
<Setter Property="Background" Value="#546A9F" />
</Style>
<Style Selector="TabItem">
<Setter Property="Foreground" Value="#E9E9E9" />
<Setter Property="FontSize" Value="20" />
</Style>
<Style Selector="TabItem[TabStripPlacement=Left] /template/ ContentPresenter#PART_ContentPresenter">
<Setter Property="Margin" Value="5 0" />
</Style>
<Style Selector="TabItem:pointerover">
<Setter Property="Background" Value="#1F2E45" />
</Style>
<Style Selector="TabItem:pointerover /template/ Border">
<Setter Property="Background" Value="#1F2E45" />
</Style>
<Style Selector="TabItem:pointerover > TextBlock">
<Setter Property="Foreground" Value="#E9E9E9" />
</Style>
<Style Selector="TabItem:selected:pointerover /template/ Border">
<Setter Property="Background" Value="#FFFFFF" />
</Style>
<Style Selector="TabItem:selected:pointerover > TextBlock">
<Setter Property="Foreground" Value="#1A2234" />
</Style>
<Style Selector="TabItem:selected">
<Setter Property="Foreground" Value="#1A2234" />
<Setter Property="Background" Value="#FFFFFF" />
</Style>
<Style Selector="TabItem:selected /template/ Border#PART_SelectedPipe">
<Setter Property="IsVisible" Value="False" />
</Style>
<Style Selector="TabItem:disabled > TextBlock">
<Setter Property="Foreground" Value="#B2B2B2" />
<Setter Property="TextDecorations" Value="Strikethrough" />
</Style>
<Style Selector="TabItem.first">
<Setter Property="Margin" Value="0 13 0 0" />
</Style>
<Style Selector="TabControl">
<Setter Property="Padding" Value="0" />
</Style>
<Style Selector="ContentPresenter.page">
<Setter Property="Margin" Value="15 21" />
</Style>
</UserControl.Styles>
<DockPanel>
<TabControl x:Name="TabControl" TabStripPlacement="Left">
<TabItem x:Name="TabDatabase" Header="Database" Classes="first">
<DockPanel>
<controls:StatusBar DataContext="{Binding StatusBarModel}" DockPanel.Dock="Bottom" />
<ScrollViewer>
<ContentPresenter Content="{Binding DatabasePage}" Classes="page" />
</ScrollViewer>
</DockPanel>
</TabItem>
<TabItem x:Name="TabTracking" Header="Tracking">
<DockPanel>
<controls:StatusBar DataContext="{Binding StatusBarModel}" DockPanel.Dock="Bottom" />
<ScrollViewer>
<ContentPresenter Content="{Binding TrackingPage}" Classes="page" />
</ScrollViewer>
</DockPanel>
</TabItem>
<TabItem x:Name="TabViewer" Header="Viewer">
<DockPanel>
<controls:StatusBar DataContext="{Binding StatusBarModel}" DockPanel.Dock="Bottom" />
<ScrollViewer>
<ContentPresenter Content="{Binding ViewerPage}" Classes="page" />
</ScrollViewer>
</DockPanel>
</TabItem>
</TabControl>
</DockPanel>
</UserControl>

2
app/Desktop/Main/Screens/MainContentScreen.axaml.cs → app/Desktop/Main/MainContentScreen.axaml.cs
View File

@@ -2,7 +2,7 @@ using System.Diagnostics.CodeAnalysis;
using Avalonia.Controls;
using Avalonia.Markup.Xaml;
namespace DHT.Desktop.Main.Screens {
namespace DHT.Desktop.Main {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class MainContentScreen : UserControl {
public MainContentScreen() {

61
app/Desktop/Main/MainContentScreenModel.cs Normal file
View File

@@ -0,0 +1,61 @@
using System;
using System.Threading.Tasks;
using Avalonia.Controls;
using DHT.Desktop.Main.Controls;
using DHT.Desktop.Main.Pages;
using DHT.Server.Database;
using DHT.Server.Service;
namespace DHT.Desktop.Main {
sealed class MainContentScreenModel : IDisposable {
public DatabasePage DatabasePage { get; }
private DatabasePageModel DatabasePageModel { get; }
public TrackingPage TrackingPage { get; }
private TrackingPageModel TrackingPageModel { get; }
public ViewerPage ViewerPage { get; }
private ViewerPageModel ViewerPageModel { get; }
public StatusBarModel StatusBarModel { get; }
public event EventHandler? DatabaseClosed {
add {
DatabasePageModel.DatabaseClosed += value;
}
remove {
DatabasePageModel.DatabaseClosed -= value;
}
}
[Obsolete("Designer")]
public MainContentScreenModel() : this(null!, DummyDatabaseFile.Instance) {}
public MainContentScreenModel(Window window, IDatabaseFile db) {
DatabasePageModel = new DatabasePageModel(window, db);
DatabasePage = new DatabasePage { DataContext = DatabasePageModel };
TrackingPageModel = new TrackingPageModel(window, db);
TrackingPage = new TrackingPage { DataContext = TrackingPageModel };
ViewerPageModel = new ViewerPageModel(window, db);
ViewerPage = new ViewerPage { DataContext = ViewerPageModel };
StatusBarModel = new StatusBarModel(db.Statistics);
TrackingPageModel.ServerStatusChanged += TrackingPageModelOnServerStatusChanged;
StatusBarModel.CurrentStatus = ServerLauncher.IsRunning ? StatusBarModel.Status.Ready : StatusBarModel.Status.Stopped;
}
public async Task Initialize() {
await TrackingPageModel.Initialize();
}
private void TrackingPageModelOnServerStatusChanged(object? sender, StatusBarModel.Status e) {
StatusBarModel.CurrentStatus = e;
}
public void Dispose() {
TrackingPageModel.Dispose();
}
}
}

5
app/Desktop/Main/MainWindow.axaml
View File

@@ -8,7 +8,7 @@
Title="{Binding Title}"
Icon="avares://DiscordHistoryTracker/Resources/icon.ico"
Width="800" Height="500"
MinWidth="520" MinHeight="300"
MinWidth="480" MinHeight="240"
WindowStartupLocation="CenterScreen"
Closed="OnClosed">
@@ -17,6 +17,7 @@
</Design.DataContext>
<Panel>
<ContentPresenter Content="{Binding CurrentScreen}" />
<ContentPresenter Content="{Binding WelcomeScreen}" IsVisible="{Binding ShowWelcomeScreen}" />
<ContentPresenter Content="{Binding MainContentScreen}" IsVisible="{Binding ShowMainContentScreen}" />
</Panel>
</Window>

10
app/Desktop/Main/MainWindow.axaml.cs
View File

@@ -1,10 +1,8 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.IO;
using Avalonia;
using Avalonia.Controls;
using Avalonia.Markup.Xaml;
using DHT.Desktop.Main.Pages;
using JetBrains.Annotations;
namespace DHT.Desktop.Main {
@@ -32,14 +30,6 @@ namespace DHT.Desktop.Main {
if (DataContext is IDisposable disposable) {
disposable.Dispose();
}
foreach (var temporaryFile in ViewerPageModel.TemporaryFiles) {
try {
File.Delete(temporaryFile);
} catch (Exception) {
// ignored
}
}
}
}
}

63
app/Desktop/Main/MainWindowModel.cs
View File

@@ -5,8 +5,7 @@ using System.Runtime.InteropServices;
using System.Threading.Tasks;
using Avalonia.Controls;
using DHT.Desktop.Dialogs.Message;
using DHT.Desktop.Main.Screens;
using DHT.Desktop.Server;
using DHT.Desktop.Main.Pages;
using DHT.Server.Database;
using DHT.Utils.Models;
@@ -16,13 +15,14 @@ namespace DHT.Desktop.Main {
public string Title { get; private set; } = DefaultTitle;
public UserControl CurrentScreen { get; private set; }
private readonly WelcomeScreen welcomeScreen;
private readonly WelcomeScreenModel welcomeScreenModel;
public WelcomeScreen WelcomeScreen { get; }
private WelcomeScreenModel WelcomeScreenModel { get; }
private MainContentScreen? mainContentScreen;
private MainContentScreenModel? mainContentScreenModel;
public MainContentScreen? MainContentScreen { get; private set; }
private MainContentScreenModel? MainContentScreenModel { get; set; }
public bool ShowWelcomeScreen => db == null;
public bool ShowMainContentScreen => db != null;
private readonly Window window;
@@ -34,11 +34,10 @@ namespace DHT.Desktop.Main {
public MainWindowModel(Window window, Arguments args) {
this.window = window;
welcomeScreenModel = new WelcomeScreenModel(window);
welcomeScreen = new WelcomeScreen { DataContext = welcomeScreenModel };
CurrentScreen = welcomeScreen;
WelcomeScreenModel = new WelcomeScreenModel(window);
WelcomeScreen = new WelcomeScreen { DataContext = WelcomeScreenModel };
welcomeScreenModel.PropertyChanged += WelcomeScreenModelOnPropertyChanged;
WelcomeScreenModel.PropertyChanged += WelcomeScreenModelOnPropertyChanged;
var dbFile = args.DatabaseFile;
if (!string.IsNullOrWhiteSpace(dbFile)) {
@@ -51,7 +50,7 @@ namespace DHT.Desktop.Main {
}
if (File.Exists(dbFile)) {
await welcomeScreenModel.OpenOrCreateDatabaseFromPath(dbFile);
await WelcomeScreenModel.OpenOrCreateDatabaseFromPath(dbFile);
}
else {
await Dialog.ShowOk(window, "Database Error", "Database file not found:\n" + dbFile);
@@ -62,40 +61,40 @@ namespace DHT.Desktop.Main {
}
if (args.ServerPort != null) {
ServerManager.Port = args.ServerPort.Value;
TrackingPageModel.ServerPort = args.ServerPort.ToString()!;
}
if (args.ServerToken != null) {
ServerManager.Token = args.ServerToken;
TrackingPageModel.ServerToken = args.ServerToken;
}
}
private async void WelcomeScreenModelOnPropertyChanged(object? sender, PropertyChangedEventArgs e) {
if (e.PropertyName == nameof(welcomeScreenModel.Db)) {
if (mainContentScreenModel != null) {
mainContentScreenModel.DatabaseClosed -= MainContentScreenModelOnDatabaseClosed;
mainContentScreenModel.Dispose();
if (e.PropertyName == nameof(WelcomeScreenModel.Db)) {
if (MainContentScreenModel != null) {
MainContentScreenModel.DatabaseClosed -= MainContentScreenModelOnDatabaseClosed;
MainContentScreenModel.Dispose();
}
db?.Dispose();
db = welcomeScreenModel.Db;
db = WelcomeScreenModel.Db;
if (db == null) {
Title = DefaultTitle;
mainContentScreenModel = null;
mainContentScreen = null;
CurrentScreen = welcomeScreen;
MainContentScreenModel = null;
MainContentScreen = null;
}
else {
Title = Path.GetFileName(db.Path) + " - " + DefaultTitle;
mainContentScreenModel = new MainContentScreenModel(window, db);
await mainContentScreenModel.Initialize();
mainContentScreenModel.DatabaseClosed += MainContentScreenModelOnDatabaseClosed;
mainContentScreen = new MainContentScreen { DataContext = mainContentScreenModel };
CurrentScreen = mainContentScreen;
MainContentScreenModel = new MainContentScreenModel(window, db);
await MainContentScreenModel.Initialize();
MainContentScreenModel.DatabaseClosed += MainContentScreenModelOnDatabaseClosed;
MainContentScreen = new MainContentScreen { DataContext = MainContentScreenModel };
OnPropertyChanged(nameof(MainContentScreen));
}
OnPropertyChanged(nameof(CurrentScreen));
OnPropertyChanged(nameof(ShowWelcomeScreen));
OnPropertyChanged(nameof(ShowMainContentScreen));
OnPropertyChanged(nameof(Title));
window.Focus();
@@ -103,14 +102,12 @@ namespace DHT.Desktop.Main {
}
private void MainContentScreenModelOnDatabaseClosed(object? sender, EventArgs e) {
welcomeScreenModel.CloseDatabase();
WelcomeScreenModel.CloseDatabase();
}
public void Dispose() {
welcomeScreenModel.Dispose();
mainContentScreenModel?.Dispose();
MainContentScreenModel?.Dispose();
db?.Dispose();
db = null;
}
}
}

25
app/Desktop/Main/Pages/AdvancedPage.axaml
View File

@@ -1,25 +0,0 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:pages="clr-namespace:DHT.Desktop.Main.Pages"
xmlns:controls="clr-namespace:DHT.Desktop.Main.Controls"
mc:Ignorable="d" d:DesignWidth="800" d:DesignHeight="450"
x:Class="DHT.Desktop.Main.Pages.AdvancedPage">
<Design.DataContext>
<pages:AdvancedPageModel />
</Design.DataContext>
<StackPanel Orientation="Vertical" Spacing="10">
<Expander Header="Internal Server Configuration" IsExpanded="True">
<controls:ServerConfigurationPanel DataContext="{Binding ServerConfigurationModel}" />
</Expander>
<Expander Header="Database Tools" IsExpanded="True">
<StackPanel Orientation="Vertical" Spacing="10">
<TextBlock TextWrapping="Wrap">Recreates the database to remove any traces of deleted data, which frees up space and prevents forensic recovery.</TextBlock>
<Button Command="{Binding VacuumDatabase}">Vacuum Database</Button>
</StackPanel>
</Expander>
</StackPanel>
</UserControl>

16
app/Desktop/Main/Pages/AdvancedPage.axaml.cs
View File

@@ -1,16 +0,0 @@
using System.Diagnostics.CodeAnalysis;
using Avalonia.Controls;
using Avalonia.Markup.Xaml;
namespace DHT.Desktop.Main.Pages {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class AdvancedPage : UserControl {
public AdvancedPage() {
InitializeComponent();
}
private void InitializeComponent() {
AvaloniaXamlLoader.Load(this);
}
}
}

39
app/Desktop/Main/Pages/AdvancedPageModel.cs
View File

@@ -1,39 +0,0 @@
using System;
using Avalonia.Controls;
using DHT.Desktop.Dialogs.Message;
using DHT.Desktop.Main.Controls;
using DHT.Desktop.Server;
using DHT.Server.Database;
using DHT.Utils.Models;
namespace DHT.Desktop.Main.Pages {
sealed class AdvancedPageModel : BaseModel, IDisposable {
public ServerConfigurationPanelModel ServerConfigurationModel { get; }
private readonly Window window;
private readonly IDatabaseFile db;
[Obsolete("Designer")]
public AdvancedPageModel() : this(null!, DummyDatabaseFile.Instance, new ServerManager(DummyDatabaseFile.Instance)) {}
public AdvancedPageModel(Window window, IDatabaseFile db, ServerManager serverManager) {
this.window = window;
this.db = db;
ServerConfigurationModel = new ServerConfigurationPanelModel(window, serverManager);
}
public void Initialize() {
ServerConfigurationModel.Initialize();
}
public void Dispose() {
ServerConfigurationModel.Dispose();
}
public async void VacuumDatabase() {
db.Vacuum();
await Dialog.ShowOk(window, "Vacuum Database", "Done.");
}
}
}

54
app/Desktop/Main/Pages/AttachmentsPage.axaml
View File

@@ -1,54 +0,0 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:pages="clr-namespace:DHT.Desktop.Main.Pages"
xmlns:controls="clr-namespace:DHT.Desktop.Main.Controls"
mc:Ignorable="d" d:DesignWidth="800" d:DesignHeight="450"
x:Class="DHT.Desktop.Main.Pages.AttachmentsPage">
<Design.DataContext>
<pages:AttachmentsPageModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="Expander">
<Setter Property="Margin" Value="0 5 0 0" />
</Style>
<Style Selector="DataGridColumnHeader">
<Setter Property="FontWeight" Value="Medium" />
</Style>
<Style Selector="DataGridColumnHeader:nth-child(2)">
<Setter Property="HorizontalContentAlignment" Value="Right" />
</Style>
<Style Selector="DataGridColumnHeader:nth-child(3)">
<Setter Property="HorizontalContentAlignment" Value="Right" />
</Style>
<Style Selector="DataGridCell.right">
<Setter Property="HorizontalContentAlignment" Value="Right" />
</Style>
</UserControl.Styles>
<StackPanel Orientation="Vertical" Spacing="20">
<DockPanel>
<Button Command="{Binding OnClickToggleDownload}" Content="{Binding ToggleDownloadButtonText}" IsEnabled="{Binding IsToggleDownloadButtonEnabled}" DockPanel.Dock="Left" />
<TextBlock Text="{Binding DownloadMessage}" Margin="10 0 0 0" VerticalAlignment="Center" DockPanel.Dock="Left" />
<ProgressBar Value="{Binding DownloadProgress}" IsVisible="{Binding IsDownloading}" Margin="15 0" VerticalAlignment="Center" DockPanel.Dock="Right" />
</DockPanel>
<controls:AttachmentFilterPanel DataContext="{Binding FilterModel}" IsEnabled="{Binding !DataContext.IsDownloading, RelativeSource={RelativeSource AncestorType=UserControl}}" />
<StackPanel Orientation="Vertical" Spacing="12">
<Expander Header="Download Status" IsExpanded="True">
<DataGrid Items="{Binding StatisticsRows}" AutoGenerateColumns="False" CanUserReorderColumns="False" CanUserResizeColumns="False" CanUserSortColumns="False" IsReadOnly="True">
<DataGrid.Columns>
<DataGridTextColumn Header="State" Binding="{Binding State}" Width="*" />
<DataGridTextColumn Header="Attachments" Binding="{Binding Items, Converter={StaticResource NumberValueConverter}}" Width="*" CellStyleClasses="right" />
<DataGridTextColumn Header="Size" Binding="{Binding Size, Converter={StaticResource BytesValueConverter}}" Width="*" CellStyleClasses="right" />
</DataGrid.Columns>
</DataGrid>
</Expander>
<StackPanel Orientation="Horizontal" Spacing="10">
<Button Command="{Binding OnClickRetryFailedDownloads}" IsEnabled="{Binding HasFailedDownloads}">Retry Failed Downloads</Button>
</StackPanel>
</StackPanel>
</StackPanel>
</UserControl>

16
app/Desktop/Main/Pages/AttachmentsPage.axaml.cs
View File

@@ -1,16 +0,0 @@
using System.Diagnostics.CodeAnalysis;
using Avalonia.Controls;
using Avalonia.Markup.Xaml;
namespace DHT.Desktop.Main.Pages {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class AttachmentsPage : UserControl {
public AttachmentsPage() {
InitializeComponent();
}
private void InitializeComponent() {
AvaloniaXamlLoader.Load(this);
}
}
}

203
app/Desktop/Main/Pages/AttachmentsPageModel.cs
View File

@@ -1,203 +0,0 @@
using System;
using System.Collections.Generic;
using System.ComponentModel;
using DHT.Desktop.Common;
using DHT.Desktop.Main.Controls;
using DHT.Server.Data;
using DHT.Server.Data.Aggregations;
using DHT.Server.Data.Filters;
using DHT.Server.Database;
using DHT.Server.Download;
using DHT.Utils.Models;
using DHT.Utils.Tasks;
namespace DHT.Desktop.Main.Pages {
sealed class AttachmentsPageModel : BaseModel, IDisposable {
private static readonly DownloadItemFilter EnqueuedItemFilter = new() {
IncludeStatuses = new HashSet<DownloadStatus> {
DownloadStatus.Enqueued
}
};
private bool isThreadDownloadButtonEnabled = true;
public string ToggleDownloadButtonText => downloadThread == null ? "Start Downloading" : "Stop Downloading";
public bool IsToggleDownloadButtonEnabled {
get => isThreadDownloadButtonEnabled;
set => Change(ref isThreadDownloadButtonEnabled, value);
}
public string DownloadMessage { get; set; } = "";
public double DownloadProgress => allItemsCount is null or 0 ? 0.0 : 100.0 * doneItemsCount / allItemsCount.Value;
public AttachmentFilterPanelModel FilterModel { get; }
private readonly StatisticsRow statisticsEnqueued = new ("Enqueued");
private readonly StatisticsRow statisticsDownloaded = new ("Downloaded");
private readonly StatisticsRow statisticsFailed = new ("Failed");
private readonly StatisticsRow statisticsSkipped = new ("Skipped");
public List<StatisticsRow> StatisticsRows {
get {
return new List<StatisticsRow> {
statisticsEnqueued,
statisticsDownloaded,
statisticsFailed,
statisticsSkipped
};
}
}
public bool IsDownloading => downloadThread != null;
public bool HasFailedDownloads => statisticsFailed.Items > 0;
private readonly IDatabaseFile db;
private readonly AsyncValueComputer<DownloadStatusStatistics>.Single downloadStatisticsComputer;
private BackgroundDownloadThread? downloadThread;
private int doneItemsCount;
private int? allItemsCount;
public AttachmentsPageModel() : this(DummyDatabaseFile.Instance) {}
public AttachmentsPageModel(IDatabaseFile db) {
this.db = db;
this.FilterModel = new AttachmentFilterPanelModel(db);
this.downloadStatisticsComputer = AsyncValueComputer<DownloadStatusStatistics>.WithResultProcessor(UpdateStatistics).WithOutdatedResults().BuildWithComputer(db.GetDownloadStatusStatistics);
this.downloadStatisticsComputer.Recompute();
db.Statistics.PropertyChanged += OnDbStatisticsChanged;
}
public void Dispose() {
db.Statistics.PropertyChanged -= OnDbStatisticsChanged;
FilterModel.Dispose();
DisposeDownloadThread();
}
private void OnDbStatisticsChanged(object? sender, PropertyChangedEventArgs e) {
if (e.PropertyName == nameof(DatabaseStatistics.TotalAttachments)) {
if (IsDownloading) {
EnqueueDownloadItems();
}
else {
downloadStatisticsComputer.Recompute();
}
}
else if (e.PropertyName == nameof(DatabaseStatistics.TotalDownloads)) {
downloadStatisticsComputer.Recompute();
}
}
private void EnqueueDownloadItems() {
var filter = FilterModel.CreateFilter();
filter.DownloadItemRule = AttachmentFilter.DownloadItemRules.OnlyNotPresent;
db.EnqueueDownloadItems(filter);
downloadStatisticsComputer.Recompute();
}
private void UpdateStatistics(DownloadStatusStatistics statusStatistics) {
var hadFailedDownloads = HasFailedDownloads;
statisticsEnqueued.Items = statusStatistics.EnqueuedCount;
statisticsEnqueued.Size = statusStatistics.EnqueuedSize;
statisticsDownloaded.Items = statusStatistics.SuccessfulCount;
statisticsDownloaded.Size = statusStatistics.SuccessfulSize;
statisticsFailed.Items = statusStatistics.FailedCount;
statisticsFailed.Size = statusStatistics.FailedSize;
statisticsSkipped.Items = statusStatistics.SkippedCount;
statisticsSkipped.Size = statusStatistics.SkippedSize;
OnPropertyChanged(nameof(StatisticsRows));
if (hadFailedDownloads != HasFailedDownloads) {
OnPropertyChanged(nameof(HasFailedDownloads));
}
allItemsCount = doneItemsCount + statisticsEnqueued.Items;
UpdateDownloadMessage();
}
private void UpdateDownloadMessage() {
DownloadMessage = IsDownloading ? doneItemsCount.Format() + " / " + (allItemsCount?.Format() ?? "?") : "";
OnPropertyChanged(nameof(DownloadMessage));
OnPropertyChanged(nameof(DownloadProgress));
}
private void DownloadThreadOnOnItemFinished(object? sender, DownloadItem e) {
++doneItemsCount;
UpdateDownloadMessage();
downloadStatisticsComputer.Recompute();
}
private void DownloadThreadOnOnServerStopped(object? sender, EventArgs e) {
downloadStatisticsComputer.Recompute();
IsToggleDownloadButtonEnabled = true;
}
public void OnClickToggleDownload() {
if (downloadThread == null) {
EnqueueDownloadItems();
downloadThread = new BackgroundDownloadThread(db);
downloadThread.OnItemFinished += DownloadThreadOnOnItemFinished;
downloadThread.OnServerStopped += DownloadThreadOnOnServerStopped;
}
else {
IsToggleDownloadButtonEnabled = false;
DisposeDownloadThread();
db.RemoveDownloadItems(EnqueuedItemFilter, FilterRemovalMode.RemoveMatching);
doneItemsCount = 0;
allItemsCount = null;
UpdateDownloadMessage();
}
OnPropertyChanged(nameof(ToggleDownloadButtonText));
OnPropertyChanged(nameof(IsDownloading));
}
public void OnClickRetryFailedDownloads() {
var allExceptFailedFilter = new DownloadItemFilter {
IncludeStatuses = new HashSet<DownloadStatus> {
DownloadStatus.Enqueued,
DownloadStatus.Success
}
};
db.RemoveDownloadItems(allExceptFailedFilter, FilterRemovalMode.KeepMatching);
if (IsDownloading) {
EnqueueDownloadItems();
}
}
private void DisposeDownloadThread() {
if (downloadThread != null) {
downloadThread.OnItemFinished -= DownloadThreadOnOnItemFinished;
downloadThread.StopThread();
}
downloadThread = null;
}
public sealed class StatisticsRow {
public string State { get; }
public int Items { get; set; }
public ulong? Size { get; set; }
public StatisticsRow(string state) {
State = state;
}
}
}
}

1
app/Desktop/Main/Pages/DatabasePage.axaml
View File

@@ -24,7 +24,6 @@
<WrapPanel>
<Button Command="{Binding OpenDatabaseFolder}">Open Database Folder</Button>
<Button Command="{Binding MergeWithDatabase}">Merge with Database(s)...</Button>
<Button Command="{Binding ImportLegacyArchive}">Import Legacy Archive(s)...</Button>
</WrapPanel>
</StackPanel>

143
app/Desktop/Main/Pages/DatabasePageModel.cs
View File

@@ -1,20 +1,14 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Avalonia.Controls;
using Avalonia.Threading;
using DHT.Desktop.Common;
using DHT.Desktop.Dialogs.Message;
using DHT.Desktop.Dialogs.Progress;
using DHT.Desktop.Dialogs.TextBox;
using DHT.Server.Data;
using DHT.Server.Database;
using DHT.Server.Database.Import;
using DHT.Server.Service;
using DHT.Utils.Logging;
using DHT.Utils.Models;
@@ -63,10 +57,6 @@ namespace DHT.Desktop.Main.Pages {
}
}
public void CloseDatabase() {
DatabaseClosed?.Invoke(this, EventArgs.Empty);
}
public async void MergeWithDatabase() {
var fileDialog = DatabaseGui.NewOpenDatabaseFileDialog();
fileDialog.Directory = Path.GetDirectoryName(Db.Path);
@@ -85,6 +75,11 @@ namespace DHT.Desktop.Main.Pages {
await progressDialog.ShowDialog(window);
}
public void CloseDatabase() {
ServerLauncher.Stop();
DatabaseClosed?.Invoke(this, EventArgs.Empty);
}
private static async Task MergeWithDatabaseFromPaths(IDatabaseFile target, string[] paths, ProgressDialog dialog, IProgressCallback callback) {
int total = paths.Length;
@@ -98,96 +93,7 @@ namespace DHT.Desktop.Main.Pages {
return DialogResult.YesNo.Yes == upgradeResult;
}
await PerformImport(target, paths, dialog, callback, "Database Merge", "Database Error", "database file", async path => {
SynchronizationContext? prevSyncContext = SynchronizationContext.Current;
SynchronizationContext.SetSynchronizationContext(new AvaloniaSynchronizationContext());
IDatabaseFile? db = await DatabaseGui.TryOpenOrCreateDatabaseFromPath(path, dialog, CheckCanUpgradeDatabase);
SynchronizationContext.SetSynchronizationContext(prevSyncContext);
if (db == null) {
return false;
}
try {
target.AddFrom(db);
return true;
} finally {
db.Dispose();
}
});
}
public async void ImportLegacyArchive() {
var fileDialog = new OpenFileDialog {
Title = "Open Legacy DHT Archive",
Directory = Path.GetDirectoryName(Db.Path),
AllowMultiple = true
};
string[]? paths = await fileDialog.ShowAsync(window);
if (paths == null || paths.Length == 0) {
return;
}
ProgressDialog progressDialog = new ProgressDialog();
progressDialog.DataContext = new ProgressDialogModel(async callback => await ImportLegacyArchiveFromPaths(Db, paths, progressDialog, callback)) {
Title = "Legacy Archive Import"
};
await progressDialog.ShowDialog(window);
}
private static async Task ImportLegacyArchiveFromPaths(IDatabaseFile target, string[] paths, ProgressDialog dialog, IProgressCallback callback) {
var fakeSnowflake = new FakeSnowflake();
await PerformImport(target, paths, dialog, callback, "Legacy Archive Import", "Legacy Archive Error", "archive file", async path => {
await using var jsonStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read);
return await LegacyArchiveImport.Read(jsonStream, target, fakeSnowflake, async servers => {
SynchronizationContext? prevSyncContext = SynchronizationContext.Current;
SynchronizationContext.SetSynchronizationContext(new AvaloniaSynchronizationContext());
Dictionary<DHT.Server.Data.Server, ulong>? result = await Dispatcher.UIThread.InvokeAsync(() => AskForServerIds(dialog, servers));
SynchronizationContext.SetSynchronizationContext(prevSyncContext);
return result;
});
});
}
private static async Task<Dictionary<DHT.Server.Data.Server, ulong>?> AskForServerIds(Window window, DHT.Server.Data.Server[] servers) {
static bool IsValidSnowflake(string value) {
return string.IsNullOrEmpty(value) || ulong.TryParse(value, out _);
}
var items = new List<TextBoxItem<DHT.Server.Data.Server>>();
foreach (var server in servers.OrderBy(static server => server.Type).ThenBy(static server => server.Name)) {
items.Add(new TextBoxItem<DHT.Server.Data.Server>(server) {
Title = server.Name + " (" + ServerTypes.ToNiceString(server.Type) + ")",
ValidityCheck = IsValidSnowflake
});
}
var model = new TextBoxDialogModel<DHT.Server.Data.Server>(items) {
Title = "Imported Server IDs",
Description = "Please fill in the IDs of servers and direct messages. First enable Developer Mode in Discord, then right-click each server or direct message, click 'Copy ID', and paste it into the input field. If a server no longer exists, leave its input field empty to use a random ID."
};
var dialog = new TextBoxDialog { DataContext = model };
var result = await dialog.ShowDialog<DialogResult.OkCancel>(window);
if (result != DialogResult.OkCancel.Ok) {
return null;
}
return model.ValidItems
.Where(static item => !string.IsNullOrEmpty(item.Value))
.ToDictionary(static item => item.Item, static item => ulong.Parse(item.Value));
}
private static async Task PerformImport(IDatabaseFile target, string[] paths, ProgressDialog dialog, IProgressCallback callback, string neutralDialogTitle, string errorDialogTitle, string itemName, Func<string, Task<bool>> performImport) {
int total = paths.Length;
var oldStatistics = target.SnapshotStatistics();
var oldStatistics = target.Statistics.Clone();
int successful = 0;
int finished = 0;
@@ -196,53 +102,56 @@ namespace DHT.Desktop.Main.Pages {
++finished;
if (!File.Exists(path)) {
await Dialog.ShowOk(dialog, errorDialogTitle, "File '" + Path.GetFileName(path) + "' no longer exists.");
await Dialog.ShowOk(dialog, "Database Error", "Database '" + Path.GetFileName(path) + "' no longer exists.");
continue;
}
IDatabaseFile? db = await DatabaseGui.TryOpenOrCreateDatabaseFromPath(path, dialog, CheckCanUpgradeDatabase);
if (db == null) {
continue;
}
try {
if (await performImport(path)) {
++successful;
}
target.AddFrom(db);
} catch (Exception ex) {
Log.Error(ex);
await Dialog.ShowOk(dialog, errorDialogTitle, "File '" + Path.GetFileName(path) + "' could not be imported: " + ex.Message);
await Dialog.ShowOk(dialog, "Database Error", "Database '" + Path.GetFileName(path) + "' could not be merged: " + ex.Message);
continue;
} finally {
db.Dispose();
}
++successful;
}
await callback.Update("Done", finished, total);
if (successful == 0) {
await Dialog.ShowOk(dialog, neutralDialogTitle, "Nothing was imported.");
await Dialog.ShowOk(dialog, "Database Merge", "Nothing was merged.");
return;
}
await Dialog.ShowOk(dialog, neutralDialogTitle, GetImportDialogMessage(oldStatistics, target.SnapshotStatistics(), successful, total, itemName));
}
private static string GetImportDialogMessage(DatabaseStatisticsSnapshot oldStatistics, DatabaseStatisticsSnapshot newStatistics, int successfulItems, int totalItems, string itemName) {
var newStatistics = target.Statistics;
long newServers = newStatistics.TotalServers - oldStatistics.TotalServers;
long newChannels = newStatistics.TotalChannels - oldStatistics.TotalChannels;
long newUsers = newStatistics.TotalUsers - oldStatistics.TotalUsers;
long newMessages = newStatistics.TotalMessages - oldStatistics.TotalMessages;
StringBuilder message = new StringBuilder();
message.Append("Processed ");
if (successfulItems == totalItems) {
message.Append(successfulItems.Pluralize(itemName));
if (successful == total) {
message.Append(successful.Pluralize("database file"));
}
else {
message.Append(successfulItems.Format()).Append(" out of ").Append(totalItems.Pluralize(itemName));
message.Append(successful.Format()).Append(" out of ").Append(total.Pluralize("database file"));
}
message.Append(" and added:\n\n \u2022 ");
message.Append(newServers.Pluralize("server")).Append("\n \u2022 ");
message.Append(newChannels.Pluralize("channel")).Append("\n \u2022 ");
message.Append(newUsers.Pluralize("user")).Append("\n \u2022 ");
message.Append(newMessages.Pluralize("message"));
return message.ToString();
await Dialog.ShowOk(dialog, "Database Merge", message.ToString());
}
}
}

45
app/Desktop/Main/Pages/DebugPage.axaml
View File

@@ -1,45 +0,0 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:pages="clr-namespace:DHT.Desktop.Main.Pages"
mc:Ignorable="d" d:DesignWidth="800" d:DesignHeight="450"
x:Class="DHT.Desktop.Main.Pages.DebugPage">
<Design.DataContext>
<pages:DebugPageModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="TextBox">
<Setter Property="FontFamily" Value="Consolas,Courier" />
<Setter Property="FontSize" Value="15" />
</Style>
<Style Selector="WrapPanel > StackPanel">
<Setter Property="Orientation" Value="Vertical" />
<Setter Property="Margin" Value="0 0 10 10" />
</Style>
</UserControl.Styles>
<StackPanel Orientation="Vertical" Spacing="10">
<Expander Header="Generate Random Data" IsExpanded="True">
<WrapPanel>
<StackPanel>
<Label Target="Channels">Channels</Label>
<TextBox x:Name="Channels" Width="100" Text="{Binding GenerateChannels}" />
</StackPanel>
<StackPanel>
<Label Target="Users">Users</Label>
<TextBox x:Name="Users" Width="100" Text="{Binding GenerateUsers}" />
</StackPanel>
<StackPanel>
<Label Target="Messages">Messages</Label>
<TextBox x:Name="Messages" Width="100" Text="{Binding GenerateMessages}" />
</StackPanel>
<StackPanel VerticalAlignment="Bottom">
<Button Command="{Binding OnClickAddRandomDataToDatabase}">Add to Database</Button>
</StackPanel>
</WrapPanel>
</Expander>
</StackPanel>
</UserControl>

16
app/Desktop/Main/Pages/DebugPage.axaml.cs
View File

@@ -1,16 +0,0 @@
using System.Diagnostics.CodeAnalysis;
using Avalonia.Controls;
using Avalonia.Markup.Xaml;
namespace DHT.Desktop.Main.Pages {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class DebugPage : UserControl {
public DebugPage() {
InitializeComponent();
}
private void InitializeComponent() {
AvaloniaXamlLoader.Load(this);
}
}
}

185
app/Desktop/Main/Pages/DebugPageModel.cs
View File

@@ -1,185 +0,0 @@
#if DEBUG
using System;
using System.Collections.Immutable;
using System.Linq;
using System.Threading.Tasks;
using Avalonia.Controls;
using DHT.Desktop.Dialogs.Message;
using DHT.Desktop.Dialogs.Progress;
using DHT.Server.Data;
using DHT.Server.Database;
using DHT.Server.Service;
using DHT.Utils.Models;
namespace DHT.Desktop.Main.Pages {
sealed class DebugPageModel : BaseModel {
public string GenerateChannels { get; set; } = "0";
public string GenerateUsers { get; set; } = "0";
public string GenerateMessages { get; set; } = "0";
private readonly Window window;
private readonly IDatabaseFile db;
[Obsolete("Designer")]
public DebugPageModel() : this(null!, DummyDatabaseFile.Instance) {}
public DebugPageModel(Window window, IDatabaseFile db) {
this.window = window;
this.db = db;
}
public async void OnClickAddRandomDataToDatabase() {
if (!int.TryParse(GenerateChannels, out int channels) || channels < 1) {
await Dialog.ShowOk(window, "Generate Random Data", "Amount of channels must be at least 1!");
return;
}
if (!int.TryParse(GenerateUsers, out int users) || users < 1) {
await Dialog.ShowOk(window, "Generate Random Data", "Amount of users must be at least 1!");
return;
}
if (!int.TryParse(GenerateMessages, out int messages) || messages < 1) {
await Dialog.ShowOk(window, "Generate Random Data", "Amount of messages must be at least 1!");
return;
}
ProgressDialog progressDialog = new ProgressDialog {
DataContext = new ProgressDialogModel(async callback => await GenerateRandomData(channels, users, messages, callback)) {
Title = "Generating Random Data"
}
};
await progressDialog.ShowDialog(window);
}
private const int BatchSize = 500;
private async Task GenerateRandomData(int channelCount, int userCount, int messageCount, IProgressCallback callback) {
int batchCount = (messageCount + BatchSize - 1) / BatchSize;
await callback.Update("Adding messages in batches of " + BatchSize, 0, batchCount);
var rand = new Random();
var server = new DHT.Server.Data.Server {
Id = RandomId(rand),
Name = RandomName("s"),
Type = ServerType.Server
};
var channels = Enumerable.Range(0, channelCount).Select(i => new Channel {
Id = RandomId(rand),
Server = server.Id,
Name = RandomName("c"),
ParentId = null,
Position = i,
Topic = RandomText(rand, 10),
Nsfw = rand.Next(4) == 0
}).ToArray();
var users = Enumerable.Range(0, userCount).Select(_ => new User {
Id = RandomId(rand),
Name = RandomName("u"),
AvatarUrl = null,
Discriminator = rand.Next(0, 9999).ToString()
}).ToArray();
db.AddServer(server);
db.AddUsers(users);
foreach (var channel in channels) {
db.AddChannel(channel);
}
var now = DateTimeOffset.Now;
int batchIndex = 0;
while (messageCount > 0) {
int hourOffset = batchIndex;
var messages = Enumerable.Range(0, Math.Min(messageCount, BatchSize)).Select(i => {
DateTimeOffset time = now.AddHours(hourOffset).AddMinutes((i * 60.0) / BatchSize);
DateTimeOffset? edit = rand.Next(100) == 0 ? time.AddSeconds(rand.Next(1, 60)) : null;
var timeMillis = time.ToUnixTimeMilliseconds();
var editMillis = edit?.ToUnixTimeMilliseconds();
return new Message {
Id = (ulong) timeMillis,
Sender = RandomBiasedIndex(rand, users).Id,
Channel = RandomBiasedIndex(rand, channels).Id,
Text = RandomText(rand, 100),
Timestamp = timeMillis,
EditTimestamp = editMillis,
RepliedToId = null,
Attachments = ImmutableArray<Attachment>.Empty,
Embeds = ImmutableArray<Embed>.Empty,
Reactions = ImmutableArray<Reaction>.Empty
};
}).ToArray();
db.AddMessages(messages);
messageCount -= BatchSize;
await callback.Update("Adding messages in batches of " + BatchSize, ++batchIndex, batchCount);
}
}
private static ulong RandomId(Random rand) {
ulong h = unchecked((ulong) rand.Next());
ulong l = unchecked((ulong) rand.Next());
return (h << 32) | l;
}
private static string RandomName(string prefix) {
return prefix + "-" + ServerUtils.GenerateRandomToken(5);
}
private static T RandomBiasedIndex<T>(Random rand, T[] options) {
return options[(int) Math.Floor(options.Length * rand.NextDouble() * rand.NextDouble())];
}
private static readonly string[] RandomWords = {
"apple", "apricot", "artichoke", "arugula", "asparagus", "avocado",
"banana", "bean", "beechnut", "beet", "blackberry", "blackcurrant", "blueberry", "boysenberry", "bramble", "broccoli",
"cabbage", "cacao", "cantaloupe", "caper", "carambola", "carrot", "cauliflower", "celery", "chard", "cherry", "chokeberry", "citron", "clementine", "coconut", "corn", "crabapple", "cranberry", "cucumber", "currant",
"daikon", "date", "dewberry", "durian",
"edamame", "eggplant", "elderberry", "endive",
"fig",
"garlic", "ginger", "gooseberry", "grape", "grapefruit", "guava",
"honeysuckle", "horseradish", "huckleberry",
"jackfruit", "jicama",
"kale", "kiwi", "kohlrabi", "kumquat",
"leek", "lemon", "lentil", "lettuce", "lime",
"mandarin", "mango", "mushroom", "myrtle",
"nectarine", "nut",
"olive", "okra", "onion", "orange",
"papaya", "parsnip", "pawpaw", "peach", "pear", "pea", "pepper", "persimmon", "pineapple", "plum", "plantain", "pomegranate", "pomelo", "potato", "prune", "pumpkin",
"quandong", "quinoa",
"radicchio", "radish", "raisin", "raspberry", "redcurrant", "rhubarb", "rutabaga",
"spinach", "strawberry", "squash",
"tamarind", "tangerine", "tomatillo", "tomato", "turnip",
"vanilla",
"watercress", "watermelon",
"yam",
"zucchini"
};
private static string RandomText(Random rand, int maxWords) {
int wordCount = 1 + (int) Math.Floor(maxWords * Math.Pow(rand.NextDouble(), 3));
return string.Join(' ', Enumerable.Range(0, wordCount).Select(_ => RandomWords[rand.Next(RandomWords.Length)]));
}
}
}
#else
using DHT.Utils.Models;
namespace DHT.Desktop.Main.Pages {
sealed class DebugPageModel : BaseModel {
public string GenerateChannels { get; set; } = "0";
public string GenerateUsers { get; set; } = "0";
public string GenerateMessages { get; set; } = "0";
public void OnClickAddRandomDataToDatabase() {}
}
}
#endif

40
app/Desktop/Main/Pages/TrackingPage.axaml
View File

@@ -10,14 +10,52 @@
<pages:TrackingPageModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="TextBox">
<Setter Property="FontFamily" Value="Consolas,Courier" />
<Setter Property="FontSize" Value="15" />
</Style>
<Style Selector="WrapPanel > StackPanel">
<Setter Property="Orientation" Value="Vertical" />
<Setter Property="Margin" Value="0 0 10 10" />
</Style>
</UserControl.Styles>
<StackPanel Spacing="10">
<TextBlock TextWrapping="Wrap">
To start tracking messages, copy the tracking script and paste it into the console of either the Discord app, or your browser. The console is usually opened by pressing Ctrl+Shift+I.
</TextBlock>
<StackPanel DockPanel.Dock="Left" Orientation="Horizontal" Spacing="10">
<Button x:Name="CopyTrackingScript" Click="CopyTrackingScriptButton_OnClick">Copy Tracking Script</Button>
<Button Command="{Binding OnClickToggleTrackingButton}" Content="{Binding ToggleTrackingButtonText}" IsEnabled="{Binding IsToggleButtonEnabled}" />
</StackPanel>
<TextBlock TextWrapping="Wrap" Margin="0 5 0 0">
<Expander Header="Advanced Tracking Settings">
<StackPanel Spacing="10">
<TextBlock TextWrapping="Wrap">
The following settings determine how the tracking script communicates with this application. If you change them, you will have to copy and apply the tracking script again.
</TextBlock>
<WrapPanel>
<StackPanel>
<Label Target="Port">Port</Label>
<TextBox x:Name="Port" Width="70" Text="{Binding InputPort}" />
</StackPanel>
<StackPanel>
<Label Target="Token">Token</Label>
<StackPanel Orientation="Horizontal">
<TextBox x:Name="Token" Width="200" Text="{Binding InputToken}" />
</StackPanel>
</StackPanel>
<StackPanel VerticalAlignment="Bottom">
<Button Command="{Binding OnClickRandomizeToken}">Randomize Token</Button>
</StackPanel>
</WrapPanel>
<StackPanel Orientation="Horizontal" Spacing="10">
<Button IsEnabled="{Binding HasMadeChanges}" Command="{Binding OnClickApplyChanges}">Apply &amp; Restart</Button>
<Button IsEnabled="{Binding HasMadeChanges}" Command="{Binding OnClickCancelChanges}">Cancel</Button>
</StackPanel>
</StackPanel>
</Expander>
<TextBlock TextWrapping="Wrap" Margin="0 15 0 0">
By default, the Discord app does not allow opening the console. The button below will change a hidden setting in the Discord app that controls whether the Ctrl+Shift+I shortcut is enabled.
</TextBlock>
<Button DockPanel.Dock="Right" Command="{Binding OnClickToggleAppDevTools}" Content="{Binding ToggleAppDevToolsButtonText}" IsEnabled="{Binding IsToggleAppDevToolsButtonEnabled}" />

125
app/Desktop/Main/Pages/TrackingPageModel.cs
View File

@@ -5,12 +5,51 @@ using Avalonia;
using Avalonia.Controls;
using DHT.Desktop.Dialogs.Message;
using DHT.Desktop.Discord;
using DHT.Desktop.Server;
using DHT.Desktop.Main.Controls;
using DHT.Server.Database;
using DHT.Server.Service;
using DHT.Utils.Logging;
using DHT.Utils.Models;
using static DHT.Desktop.Program;
namespace DHT.Desktop.Main.Pages {
sealed class TrackingPageModel : BaseModel {
sealed class TrackingPageModel : BaseModel, IDisposable {
private static readonly Log Log = Log.ForType<TrackingPageModel>();
internal static string ServerPort { get; set; } = ServerUtils.FindAvailablePort(50000, 60000).ToString();
internal static string ServerToken { get; set; } = ServerUtils.GenerateRandomToken(20);
private string inputPort = ServerPort;
public string InputPort {
get => inputPort;
set {
Change(ref inputPort, value);
OnPropertyChanged(nameof(HasMadeChanges));
}
}
private string inputToken = ServerToken;
public string InputToken {
get => inputToken;
set {
Change(ref inputToken, value);
OnPropertyChanged(nameof(HasMadeChanges));
}
}
public bool HasMadeChanges => ServerPort != InputPort || ServerToken != InputToken;
private bool isToggleTrackingButtonEnabled = true;
public bool IsToggleButtonEnabled {
get => isToggleTrackingButtonEnabled;
set => Change(ref isToggleTrackingButtonEnabled, value);
}
public string ToggleTrackingButtonText => ServerLauncher.IsRunning ? "Pause Tracking" : "Resume Tracking";
private bool areDevToolsEnabled;
private bool AreDevToolsEnabled {
@@ -33,16 +72,28 @@ namespace DHT.Desktop.Main.Pages {
}
}
public event EventHandler<StatusBarModel.Status>? ServerStatusChanged;
private readonly Window window;
private readonly IDatabaseFile db;
[Obsolete("Designer")]
public TrackingPageModel() : this(null!) {}
public TrackingPageModel() : this(null!, DummyDatabaseFile.Instance) {}
public TrackingPageModel(Window window) {
public TrackingPageModel(Window window, IDatabaseFile db) {
this.window = window;
this.db = db;
}
public async Task Initialize() {
ServerLauncher.ServerStatusChanged += ServerLauncherOnServerStatusChanged;
ServerLauncher.ServerManagementExceptionCaught += ServerLauncherOnServerManagementExceptionCaught;
if (int.TryParse(ServerPort, out int port)) {
string token = ServerToken;
ServerLauncher.Relaunch(port, token, db);
}
bool? devToolsEnabled = await DiscordAppSettings.AreDevToolsEnabled();
if (devToolsEnabled.HasValue) {
AreDevToolsEnabled = devToolsEnabled.Value;
@@ -53,10 +104,55 @@ namespace DHT.Desktop.Main.Pages {
}
}
public void Dispose() {
ServerLauncher.ServerManagementExceptionCaught -= ServerLauncherOnServerManagementExceptionCaught;
ServerLauncher.ServerStatusChanged -= ServerLauncherOnServerStatusChanged;
ServerLauncher.Stop();
}
private void ServerLauncherOnServerStatusChanged(object? sender, EventArgs e) {
ServerStatusChanged?.Invoke(this, ServerLauncher.IsRunning ? StatusBarModel.Status.Ready : StatusBarModel.Status.Stopped);
OnPropertyChanged(nameof(ToggleTrackingButtonText));
IsToggleButtonEnabled = true;
}
private async void ServerLauncherOnServerManagementExceptionCaught(object? sender, Exception ex) {
Log.Error(ex);
await Dialog.ShowOk(window, "Server Error", ex.Message);
}
private async Task<bool> StartServer() {
if (!int.TryParse(InputPort, out int port) || port is < 0 or > 65535) {
await Dialog.ShowOk(window, "Invalid Port", "Port must be a number between 0 and 65535.");
return false;
}
IsToggleButtonEnabled = false;
ServerStatusChanged?.Invoke(this, StatusBarModel.Status.Starting);
ServerLauncher.Relaunch(port, InputToken, db);
return true;
}
private void StopServer() {
IsToggleButtonEnabled = false;
ServerStatusChanged?.Invoke(this, StatusBarModel.Status.Stopping);
ServerLauncher.Stop();
}
public async Task<bool> OnClickToggleTrackingButton() {
if (ServerLauncher.IsRunning) {
StopServer();
return true;
}
else {
return await StartServer();
}
}
public async Task<bool> OnClickCopyTrackingScript() {
string bootstrap = await Resources.ReadTextAsync("Tracker/bootstrap.js");
string script = bootstrap.Replace("= 0; /*[PORT]*/", "= " + ServerManager.Port + ";")
.Replace("/*[TOKEN]*/", HttpUtility.JavaScriptStringEncode(ServerManager.Token))
string script = bootstrap.Replace("= 0; /*[PORT]*/", "= " + ServerPort + ";")
.Replace("/*[TOKEN]*/", HttpUtility.JavaScriptStringEncode(ServerToken))
.Replace("/*[IMPORTS]*/", await Resources.ReadJoinedAsync("Tracker/scripts/", '\n'))
.Replace("/*[CSS-CONTROLLER]*/", await Resources.ReadTextAsync("Tracker/styles/controller.css"))
.Replace("/*[CSS-SETTINGS]*/", await Resources.ReadTextAsync("Tracker/styles/settings.css"));
@@ -76,6 +172,23 @@ namespace DHT.Desktop.Main.Pages {
}
}
public void OnClickRandomizeToken() {
InputToken = ServerUtils.GenerateRandomToken(20);
}
public async void OnClickApplyChanges() {
if (await StartServer()) {
ServerPort = InputPort;
ServerToken = InputToken;
OnPropertyChanged(nameof(HasMadeChanges));
}
}
public void OnClickCancelChanges() {
InputPort = ServerPort;
InputToken = ServerToken;
}
public async void OnClickToggleAppDevTools() {
const string DialogTitle = "Discord App Settings File";

7
app/Desktop/Main/Pages/ViewerPage.axaml
View File

@@ -13,16 +13,17 @@
<UserControl.Styles>
<Style Selector="Expander">
<Setter Property="Margin" Value="0 5 0 0" />
<Setter Property="Margin" Value="0 25 0 0" />
</Style>
</UserControl.Styles>
<StackPanel Orientation="Vertical" Spacing="20">
<StackPanel Orientation="Vertical">
<StackPanel Orientation="Horizontal" VerticalAlignment="Top">
<Button Command="{Binding OnClickOpenViewer}" Margin="0 0 5 0">Open Viewer</Button>
<Button Command="{Binding OnClickSaveViewer}" Margin="5 0 0 0">Save Viewer</Button>
</StackPanel>
<controls:MessageFilterPanel DataContext="{Binding FilterModel}" />
<TextBlock Text="{Binding ExportedMessageText}" Margin="0 20 0 0" />
<controls:FilterPanel DataContext="{Binding FilterModel}" />
<Expander Header="Database Tools">
<StackPanel Orientation="Vertical" Spacing="10">
<StackPanel Orientation="Vertical" Spacing="4">

83
app/Desktop/Main/Pages/ViewerPageModel.cs
View File

@@ -1,10 +1,8 @@
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.ComponentModel;
using System.Diagnostics;
using System.IO;
using System.Text;
using System.Threading.Tasks;
using System.Web;
using Avalonia.Controls;
@@ -18,9 +16,9 @@ using DHT.Utils.Models;
using static DHT.Desktop.Program;
namespace DHT.Desktop.Main.Pages {
sealed class ViewerPageModel : BaseModel, IDisposable {
public static readonly ConcurrentBag<string> TemporaryFiles = new ();
sealed class ViewerPageModel : BaseModel {
public string ExportedMessageText { get; private set; } = "";
public bool DatabaseToolFilterModeKeep { get; set; } = true;
public bool DatabaseToolFilterModeRemove { get; set; } = false;
@@ -31,7 +29,7 @@ namespace DHT.Desktop.Main.Pages {
set => Change(ref hasFilters, value);
}
private MessageFilterPanelModel FilterModel { get; }
private FilterPanelModel FilterModel { get; }
private readonly Window window;
private readonly IDatabaseFile db;
@@ -43,53 +41,36 @@ namespace DHT.Desktop.Main.Pages {
this.window = window;
this.db = db;
FilterModel = new MessageFilterPanelModel(window, db, "Will export");
FilterModel.FilterPropertyChanged += OnFilterPropertyChanged;
}
public void Dispose() {
FilterModel.Dispose();
this.FilterModel = new FilterPanelModel(window, db);
this.FilterModel.FilterPropertyChanged += OnFilterPropertyChanged;
this.db.Statistics.PropertyChanged += OnDbStatisticsChanged;
UpdateStatistics();
}
private void OnFilterPropertyChanged(object? sender, PropertyChangedEventArgs e) {
UpdateStatistics();
HasFilters = FilterModel.HasAnyFilters;
}
private async Task WriteViewerFile(string path) {
const string ArchiveTag = "/*[ARCHIVE]*/";
string indexFile = await Resources.ReadTextAsync("Viewer/index.html");
string viewerTemplate = indexFile.Replace("/*[JS]*/", await Resources.ReadJoinedAsync("Viewer/scripts/", '\n'))
.Replace("/*[CSS]*/", await Resources.ReadJoinedAsync("Viewer/styles/", '\n'));
int viewerArchiveTagStart = viewerTemplate.IndexOf(ArchiveTag);
int viewerArchiveTagEnd = viewerArchiveTagStart + ArchiveTag.Length;
string jsonTempFile = path + ".tmp";
await using (var jsonStream = new FileStream(jsonTempFile, FileMode.Create, FileAccess.ReadWrite, FileShare.Read)) {
await ViewerJsonExport.Generate(jsonStream, db, FilterModel.CreateFilter());
char[] jsonBuffer = new char[Math.Min(32768, jsonStream.Position)];
jsonStream.Position = 0;
await using (var outputStream = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.Read))
await using (var outputWriter = new StreamWriter(outputStream, Encoding.UTF8)) {
await outputWriter.WriteAsync(viewerTemplate[..viewerArchiveTagStart]);
using (var jsonReader = new StreamReader(jsonStream, Encoding.UTF8)) {
int readBytes;
while ((readBytes = await jsonReader.ReadAsync(jsonBuffer, 0, jsonBuffer.Length)) > 0) {
string jsonChunk = new string(jsonBuffer, 0, readBytes);
await outputWriter.WriteAsync(HttpUtility.JavaScriptStringEncode(jsonChunk));
}
}
await outputWriter.WriteAsync(viewerTemplate[viewerArchiveTagEnd..]);
}
private void OnDbStatisticsChanged(object? sender, PropertyChangedEventArgs e) {
if (e.PropertyName == nameof(DatabaseStatistics.TotalMessages)) {
UpdateStatistics();
}
}
File.Delete(jsonTempFile);
private void UpdateStatistics() {
ExportedMessageText = "Will export " + db.CountMessages(FilterModel.CreateFilter()).Format() + " out of " + db.Statistics.TotalMessages.Format() + " message(s).";
OnPropertyChanged(nameof(ExportedMessageText));
}
private async Task<string> GenerateViewerContents() {
string json = ViewerJsonExport.Generate(db, FilterModel.CreateFilter());
string index = await Resources.ReadTextAsync("Viewer/index.html");
string viewer = index.Replace("/*[JS]*/", await Resources.ReadJoinedAsync("Viewer/scripts/", '\n'))
.Replace("/*[CSS]*/", await Resources.ReadJoinedAsync("Viewer/styles/", '\n'))
.Replace("/*[ARCHIVE]*/", HttpUtility.JavaScriptStringEncode(json));
return viewer;
}
public async void OnClickOpenViewer() {
@@ -103,10 +84,8 @@ namespace DHT.Desktop.Main.Pages {
fullPath = Path.Combine(rootPath, filenameBase + "-" + counter + ".html");
}
TemporaryFiles.Add(fullPath);
Directory.CreateDirectory(rootPath);
await WriteViewerFile(fullPath);
await File.WriteAllTextAsync(fullPath, await GenerateViewerContents());
Process.Start(new ProcessStartInfo(fullPath) { UseShellExecute = true });
}
@@ -114,7 +93,7 @@ namespace DHT.Desktop.Main.Pages {
public async void OnClickSaveViewer() {
var dialog = new SaveFileDialog {
Title = "Save Viewer",
InitialFileName = Path.GetFileNameWithoutExtension(db.Path) + ".html",
InitialFileName = "archive.html",
Directory = Path.GetDirectoryName(db.Path),
Filters = new List<FileDialogFilter> {
new() {
@@ -126,7 +105,7 @@ namespace DHT.Desktop.Main.Pages {
string? path = await dialog;
if (!string.IsNullOrEmpty(path)) {
await WriteViewerFile(path);
await File.WriteAllTextAsync(path, await GenerateViewerContents());
}
}
@@ -135,12 +114,12 @@ namespace DHT.Desktop.Main.Pages {
if (DatabaseToolFilterModeKeep) {
if (DialogResult.YesNo.Yes == await Dialog.ShowYesNo(window, "Keep Matching Messages in This Database", db.CountMessages(filter).Pluralize("message") + " will be kept, and the rest will be removed from this database. This action cannot be undone. Proceed?")) {
db.RemoveMessages(filter, FilterRemovalMode.KeepMatching);
db.RemoveMessages(filter, MessageFilterRemovalMode.KeepMatching);
}
}
else if (DatabaseToolFilterModeRemove) {
if (DialogResult.YesNo.Yes == await Dialog.ShowYesNo(window, "Remove Matching Messages in This Database", db.CountMessages(filter).Pluralize("message") + " will be removed from this database. This action cannot be undone. Proceed?")) {
db.RemoveMessages(filter, FilterRemovalMode.RemoveMatching);
db.RemoveMessages(filter, MessageFilterRemovalMode.RemoveMatching);
}
}
}

119
app/Desktop/Main/Screens/MainContentScreen.axaml
View File

@@ -1,119 +0,0 @@
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:controls="clr-namespace:DHT.Desktop.Main.Controls"
xmlns:screens="clr-namespace:DHT.Desktop.Main.Screens"
mc:Ignorable="d" d:DesignWidth="800" d:DesignHeight="450"
x:Class="DHT.Desktop.Main.Screens.MainContentScreen">
<Design.DataContext>
<screens:MainContentScreenModel />
</Design.DataContext>
<UserControl.Styles>
<Style Selector="Border.statusBar">
<Setter Property="Background" Value="#3C4F79" />
</Style>
<Style Selector="TextBlock.invisibleTabItem">
<Setter Property="FontSize" Value="20" />
<Setter Property="FontWeight" Value="SemiLight" />
<Setter Property="Margin" Value="17 0" />
<Setter Property="Opacity" Value="0" />
</Style>
<Style Selector="TabControl ItemsPresenter > Grid">
<Setter Property="Background" Value="#546A9F" />
</Style>
<Style Selector="TabItem">
<Setter Property="Cursor" Value="Hand" />
<Setter Property="Foreground" Value="#E9E9E9" />
<Setter Property="FontSize" Value="20" />
</Style>
<Style Selector="TabItem[TabStripPlacement=Left] /template/ ContentPresenter#PART_ContentPresenter">
<Setter Property="Margin" Value="5 0" />
</Style>
<Style Selector="TabItem:pointerover">
<Setter Property="Cursor" Value="Hand" />
<Setter Property="Background" Value="#455785" />
</Style>
<Style Selector="TabItem:pointerover /template/ Border">
<Setter Property="Background" Value="#455785" />
</Style>
<Style Selector="TabItem:pointerover > TextBlock">
<Setter Property="Foreground" Value="#E9E9E9" />
</Style>
<Style Selector="TabItem:selected:pointerover /template/ Border">
<Setter Property="Background" Value="#FFFFFF" />
</Style>
<Style Selector="TabItem:selected:pointerover > TextBlock">
<Setter Property="Foreground" Value="#1A2234" />
</Style>
<Style Selector="TabItem:selected">
<Setter Property="Foreground" Value="#1A2234" />
<Setter Property="Background" Value="#FFFFFF" />
</Style>
<Style Selector="TabItem:selected /template/ Border#PART_SelectedPipe">
<Setter Property="IsVisible" Value="False" />
</Style>
<Style Selector="TabItem:disabled > TextBlock">
<Setter Property="Foreground" Value="#B2B2B2" />
<Setter Property="TextDecorations" Value="Strikethrough" />
</Style>
<Style Selector="TabItem.first">
<Setter Property="Margin" Value="0 13 0 0" />
</Style>
<Style Selector="TabControl">
<Setter Property="Padding" Value="0" />
</Style>
<Style Selector="ContentPresenter.page">
<Setter Property="Margin" Value="15 21 20 21" />
</Style>
</UserControl.Styles>
<DockPanel>
<Border Classes="statusBar" DockPanel.Dock="Bottom">
<DockPanel>
<TextBlock Classes="invisibleTabItem" DockPanel.Dock="Left">Attachments</TextBlock>
<controls:StatusBar DataContext="{Binding StatusBarModel}" DockPanel.Dock="Right" />
</DockPanel>
</Border>
<TabControl x:Name="TabControl" TabStripPlacement="Left" DockPanel.Dock="Top">
<TabControl.ItemsPanel>
<ItemsPanelTemplate>
<Grid ColumnDefinitions="Auto" RowDefinitions="Auto,Auto,Auto,Auto,*,Auto,Auto" />
</ItemsPanelTemplate>
</TabControl.ItemsPanel>
<TabItem x:Name="TabDatabase" Header="Database" Classes="first" Grid.Row="0">
<ScrollViewer>
<ContentPresenter Content="{Binding DatabasePage}" Classes="page" />
</ScrollViewer>
</TabItem>
<TabItem x:Name="TabTracking" Header="Tracking" Grid.Row="1">
<ScrollViewer>
<ContentPresenter Content="{Binding TrackingPage}" Classes="page" />
</ScrollViewer>
</TabItem>
<TabItem x:Name="TabAttachments" Header="Attachments" Grid.Row="2">
<ScrollViewer>
<ContentPresenter Content="{Binding AttachmentsPage}" Classes="page" />
</ScrollViewer>
</TabItem>
<TabItem x:Name="TabViewer" Header="Viewer" Grid.Row="3">
<ScrollViewer>
<ContentPresenter Content="{Binding ViewerPage}" Classes="page" />
</ScrollViewer>
</TabItem>
<TabItem x:Name="TabAdvanced" Header="Advanced" Grid.Row="5">
<ScrollViewer>
<ContentPresenter Content="{Binding AdvancedPage}" Classes="page" />
</ScrollViewer>
</TabItem>
<TabItem x:Name="TabDebug" Header="Debug" Grid.Row="6" IsVisible="{Binding HasDebugPage}">
<ScrollViewer>
<ContentPresenter Content="{Binding DebugPage}" Classes="page" />
</ScrollViewer>
</TabItem>
</TabControl>
</DockPanel>
</UserControl>

119
app/Desktop/Main/Screens/MainContentScreenModel.cs
View File

@@ -1,119 +0,0 @@
using System;
using System.Threading.Tasks;
using Avalonia.Controls;
using DHT.Desktop.Dialogs.Message;
using DHT.Desktop.Main.Controls;
using DHT.Desktop.Main.Pages;
using DHT.Desktop.Server;
using DHT.Server.Database;
using DHT.Server.Service;
using DHT.Utils.Logging;
namespace DHT.Desktop.Main.Screens {
sealed class MainContentScreenModel : IDisposable {
private static readonly Log Log = Log.ForType<MainContentScreenModel>();
public DatabasePage DatabasePage { get; }
private DatabasePageModel DatabasePageModel { get; }
public TrackingPage TrackingPage { get; }
private TrackingPageModel TrackingPageModel { get; }
public AttachmentsPage AttachmentsPage { get; }
private AttachmentsPageModel AttachmentsPageModel { get; }
public ViewerPage ViewerPage { get; }
private ViewerPageModel ViewerPageModel { get; }
public AdvancedPage AdvancedPage { get; }
private AdvancedPageModel AdvancedPageModel { get; }
public DebugPage? DebugPage { get; }
#if DEBUG
public bool HasDebugPage => true;
private DebugPageModel DebugPageModel { get; }
#else
public bool HasDebugPage => false;
#endif
public StatusBarModel StatusBarModel { get; }
public event EventHandler? DatabaseClosed {
add {
DatabasePageModel.DatabaseClosed += value;
}
remove {
DatabasePageModel.DatabaseClosed -= value;
}
}
private readonly Window window;
private readonly ServerManager serverManager;
[Obsolete("Designer")]
public MainContentScreenModel() : this(null!, DummyDatabaseFile.Instance) {}
public MainContentScreenModel(Window window, IDatabaseFile db) {
this.window = window;
this.serverManager = new ServerManager(db);
ServerLauncher.ServerManagementExceptionCaught += ServerLauncherOnServerManagementExceptionCaught;
DatabasePageModel = new DatabasePageModel(window, db);
DatabasePage = new DatabasePage { DataContext = DatabasePageModel };
TrackingPageModel = new TrackingPageModel(window);
TrackingPage = new TrackingPage { DataContext = TrackingPageModel };
AttachmentsPageModel = new AttachmentsPageModel(db);
AttachmentsPage = new AttachmentsPage { DataContext = AttachmentsPageModel };
ViewerPageModel = new ViewerPageModel(window, db);
ViewerPage = new ViewerPage { DataContext = ViewerPageModel };
AdvancedPageModel = new AdvancedPageModel(window, db, serverManager);
AdvancedPage = new AdvancedPage { DataContext = AdvancedPageModel };
#if DEBUG
DebugPageModel = new DebugPageModel(window, db);
DebugPage = new DebugPage { DataContext = DebugPageModel };
#else
DebugPage = null;
#endif
StatusBarModel = new StatusBarModel(db.Statistics);
AdvancedPageModel.ServerConfigurationModel.ServerStatusChanged += OnServerStatusChanged;
DatabaseClosed += OnDatabaseClosed;
StatusBarModel.CurrentStatus = serverManager.IsRunning ? StatusBarModel.Status.Ready : StatusBarModel.Status.Stopped;
}
public async Task Initialize() {
await TrackingPageModel.Initialize();
AdvancedPageModel.Initialize();
serverManager.Launch();
}
public void Dispose() {
ServerLauncher.ServerManagementExceptionCaught -= ServerLauncherOnServerManagementExceptionCaught;
AttachmentsPageModel.Dispose();
ViewerPageModel.Dispose();
serverManager.Dispose();
}
private void OnServerStatusChanged(object? sender, StatusBarModel.Status e) {
StatusBarModel.CurrentStatus = e;
}
private void OnDatabaseClosed(object? sender, EventArgs e) {
serverManager.Stop();
}
private async void ServerLauncherOnServerManagementExceptionCaught(object? sender, Exception ex) {
Log.Error(ex);
await Dialog.ShowOk(window, "Internal Server Error", ex.Message);
}
}
}

6
app/Desktop/Main/Screens/WelcomeScreen.axaml → app/Desktop/Main/WelcomeScreen.axaml
View File

@@ -2,12 +2,12 @@
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:screens="clr-namespace:DHT.Desktop.Main.Screens"
xmlns:main="clr-namespace:DHT.Desktop.Main"
mc:Ignorable="d" d:DesignWidth="800" d:DesignHeight="450"
x:Class="DHT.Desktop.Main.Screens.WelcomeScreen">
x:Class="DHT.Desktop.Main.WelcomeScreen">
<Design.DataContext>
<screens:WelcomeScreenModel />
<main:WelcomeScreenModel />
</Design.DataContext>
<UserControl.Background>

2
app/Desktop/Main/Screens/WelcomeScreen.axaml.cs → app/Desktop/Main/WelcomeScreen.axaml.cs
View File

@@ -2,7 +2,7 @@ using System.Diagnostics.CodeAnalysis;
using Avalonia.Controls;
using Avalonia.Markup.Xaml;
namespace DHT.Desktop.Main.Screens {
namespace DHT.Desktop.Main {
[SuppressMessage("ReSharper", "MemberCanBeInternal")]
public sealed class WelcomeScreen : UserControl {
public WelcomeScreen() {

12
app/Desktop/Main/Screens/WelcomeScreenModel.cs → app/Desktop/Main/WelcomeScreenModel.cs
View File

@@ -7,8 +7,8 @@ using DHT.Desktop.Dialogs.Message;
using DHT.Server.Database;
using DHT.Utils.Models;
namespace DHT.Desktop.Main.Screens {
sealed class WelcomeScreenModel : BaseModel, IDisposable {
namespace DHT.Desktop.Main {
sealed class WelcomeScreenModel : BaseModel {
public string Version => Program.Version;
public IDatabaseFile? Db { get; private set; }
@@ -52,7 +52,8 @@ namespace DHT.Desktop.Main.Screens {
}
public void CloseDatabase() {
Dispose();
Db = null;
OnPropertyChanged(nameof(Db));
OnPropertyChanged(nameof(HasDatabase));
}
@@ -64,10 +65,5 @@ namespace DHT.Desktop.Main.Screens {
public void Exit() {
window.Close();
}
public void Dispose() {
Db?.Dispose();
Db = null;
}
}
}

50
app/Desktop/Server/ServerManager.cs
View File

@@ -1,50 +0,0 @@
using System;
using DHT.Server.Database;
using DHT.Server.Service;
namespace DHT.Desktop.Server {
sealed class ServerManager : IDisposable {
public static ushort Port { get; set; } = ServerUtils.FindAvailablePort(50000, 60000);
public static string Token { get; set; } = ServerUtils.GenerateRandomToken(20);
private static ServerManager? instance;
public bool IsRunning => ServerLauncher.IsRunning;
private readonly IDatabaseFile db;
public ServerManager(IDatabaseFile db) {
if (db != DummyDatabaseFile.Instance) {
if (instance != null) {
throw new InvalidOperationException("Only one instance of ServerManager can exist at the same time!");
}
instance = this;
}
this.db = db;
}
public void Launch() {
ServerLauncher.Relaunch(Port, Token, db);
}
public void Relaunch(ushort port, string token) {
Port = port;
Token = token;
Launch();
}
public void Stop() {
ServerLauncher.Stop();
}
public void Dispose() {
Stop();
if (instance == this) {
instance = null;
}
}
}
}

16
app/Resources/Tracker/bootstrap.js vendored
View File

@@ -42,10 +42,6 @@
stopTrackingDelayed(() => isSending = false);
};
const isNoAction = function(action) {
return action === null || action === CONSTANTS.AUTOSCROLL_ACTION_NOTHING;
};
const onTrackingContinued = function(anyNewMessages) {
if (!STATE.isTracking()) {
return;
@@ -63,14 +59,14 @@
if (SETTINGS.autoscroll) {
let action = null;
if (!DISCORD.hasMoreMessages()) {
action = SETTINGS.afterFirstMsg;
}
if (isNoAction(action) && !anyNewMessages) {
if (!anyNewMessages) {
action = SETTINGS.afterSavedMsg;
}
else if (!DISCORD.hasMoreMessages()) {
action = SETTINGS.afterFirstMsg;
}
if (isNoAction(action)) {
if (action === null || action === CONSTANTS.AUTOSCROLL_ACTION_NOTHING) {
DISCORD.loadOlderMessages();
}
else if (action === CONSTANTS.AUTOSCROLL_ACTION_PAUSE || (action === CONSTANTS.AUTOSCROLL_ACTION_SWITCH && !DISCORD.selectNextTextChannel())) {
@@ -117,8 +113,8 @@
try {
if (!messages.length) {
DISCORD.loadOlderMessages();
isSending = false;
onTrackingContinued(false);
}
else {
const anyNewMessages = await STATE.addDiscordMessages(info.id, messages);

1
app/Resources/Tracker/scripts.min/discord.js Normal file
View File

File diff suppressed because one or more lines are too long

1
app/Resources/Tracker/scripts.min/dom.js Normal file
View File

@@ -0,0 +1 @@
class DOM{static id(id,parent){return(parent||document).getElementById(id)}static queryReactClass(cls,parent){return(parent||document).querySelector(`[class*="${cls}-"]`)}static createElement(tag,parent,id,html){const ele=document.createElement(tag);ele.id=id||"";ele.innerHTML=html||"";parent.appendChild(ele);return ele}static removeElement(ele){return ele.parentNode.removeChild(ele)}static createStyle(styles){return this.createElement("style",document.head,"",styles)}static saveToCookie(name,obj,expiresInSeconds){const expires=new Date(Date.now()+1e3*expiresInSeconds).toUTCString();document.cookie=name+"="+encodeURIComponent(JSON.stringify(obj))+";path=/;expires="+expires}static loadFromCookie(name){const value=document.cookie.replace(new RegExp("(?:(?:^|.*;\\s*)"+name+"\\s*\\=\\s*([^;]*).*$)|^.*$"),"$1");return value.length?JSON.parse(decodeURIComponent(value)):null}static getReactProps(ele){const keys=Object.keys(ele||{});let key=keys.find(key=>key.startsWith("__reactInternalInstance"));if(key){return ele[key].memoizedProps}key=keys.find(key=>key.startsWith("__reactProps$"));return key?ele[key]:null}}

17
app/Resources/Tracker/scripts.min/gui.js Normal file
View File

@@ -0,0 +1,17 @@
const GUI=function(){let controller=null;let settings=null;const stateChangedEvent=()=>{if(settings){settings.ui.cbAutoscroll.checked=SETTINGS.autoscroll;settings.ui.optsAfterFirstMsg[SETTINGS.afterFirstMsg].checked=true;settings.ui.optsAfterSavedMsg[SETTINGS.afterSavedMsg].checked=true;const autoscrollDisabled=!SETTINGS.autoscroll;Object.values(settings.ui.optsAfterFirstMsg).forEach(ele=>ele.disabled=autoscrollDisabled);Object.values(settings.ui.optsAfterSavedMsg).forEach(ele=>ele.disabled=autoscrollDisabled)}};return{showController(){if(controller){return}const html=`
<button id='dht-ctrl-close'>X</button>
<button id='dht-ctrl-settings'>Settings</button>
<button id='dht-ctrl-track'></button>
<p id='dht-ctrl-status'>Waiting</p>`;controller={styles:DOM.createStyle(`/*[CSS-CONTROLLER]*/`),ele:DOM.createElement("div",document.body,"dht-ctrl",html)};controller.ui={btnSettings:DOM.id("dht-ctrl-settings"),btnTrack:DOM.id("dht-ctrl-track"),btnClose:DOM.id("dht-ctrl-close"),textStatus:DOM.id("dht-ctrl-status")};controller.ui.btnSettings.addEventListener("click",()=>{this.showSettings()});controller.ui.btnTrack.addEventListener("click",()=>{const isTracking=!STATE.isTracking();STATE.setIsTracking(isTracking);if(!isTracking){controller.ui.textStatus.innerText="Stopped"}});controller.ui.btnClose.addEventListener("click",()=>{this.hideController();window.DHT_ON_UNLOAD.forEach(f=>f());window.DHT_LOADED=false});STATE.onTrackingStateChanged(isTracking=>{controller.ui.btnTrack.innerText=isTracking?"Pause Tracking":"Start Tracking";controller.ui.btnSettings.disabled=isTracking});SETTINGS.onSettingsChanged(stateChangedEvent);stateChangedEvent()},hideController(){if(controller){DOM.removeElement(controller.ele);DOM.removeElement(controller.styles);controller=null}},showSettings(){if(settings){return}const radio=(type,id,label)=>"<label><input id='dht-cfg-"+type+"-"+id+"' name='dht-"+type+"' type='radio'> "+label+"</label><br>";const html=`
<label><input id='dht-cfg-autoscroll' type='checkbox'> Autoscroll</label><br>
<br>
<label>After reaching the first message in channel...</label><br>
${radio("afm","nothing","Do Nothing")}
${radio("afm","pause","Pause Tracking")}
${radio("afm","switch","Switch to Next Channel")}
<br>
<label>After reaching a previously saved message...</label><br>
${radio("asm","nothing","Do Nothing")}
${radio("asm","pause","Pause Tracking")}
${radio("asm","switch","Switch to Next Channel")}
<p id='dht-cfg-note'>It is recommended to disable link and image previews to avoid putting unnecessary strain on your browser.</p>`;settings={styles:DOM.createStyle(`/*[CSS-SETTINGS]*/`),overlay:DOM.createElement("div",document.body,"dht-cfg-overlay"),ele:DOM.createElement("div",document.body,"dht-cfg",html)};settings.overlay.addEventListener("click",()=>{this.hideSettings()});settings.ui={cbAutoscroll:DOM.id("dht-cfg-autoscroll"),optsAfterFirstMsg:{},optsAfterSavedMsg:{}};settings.ui.optsAfterFirstMsg[CONSTANTS.AUTOSCROLL_ACTION_NOTHING]=DOM.id("dht-cfg-afm-nothing");settings.ui.optsAfterFirstMsg[CONSTANTS.AUTOSCROLL_ACTION_PAUSE]=DOM.id("dht-cfg-afm-pause");settings.ui.optsAfterFirstMsg[CONSTANTS.AUTOSCROLL_ACTION_SWITCH]=DOM.id("dht-cfg-afm-switch");settings.ui.optsAfterSavedMsg[CONSTANTS.AUTOSCROLL_ACTION_NOTHING]=DOM.id("dht-cfg-asm-nothing");settings.ui.optsAfterSavedMsg[CONSTANTS.AUTOSCROLL_ACTION_PAUSE]=DOM.id("dht-cfg-asm-pause");settings.ui.optsAfterSavedMsg[CONSTANTS.AUTOSCROLL_ACTION_SWITCH]=DOM.id("dht-cfg-asm-switch");settings.ui.cbAutoscroll.addEventListener("change",()=>{SETTINGS.autoscroll=settings.ui.cbAutoscroll.checked});Object.keys(settings.ui.optsAfterFirstMsg).forEach(key=>{settings.ui.optsAfterFirstMsg[key].addEventListener("click",()=>{SETTINGS.afterFirstMsg=key})});Object.keys(settings.ui.optsAfterSavedMsg).forEach(key=>{settings.ui.optsAfterSavedMsg[key].addEventListener("click",()=>{SETTINGS.afterSavedMsg=key})});stateChangedEvent()},hideSettings(){if(settings){DOM.removeElement(settings.overlay);DOM.removeElement(settings.ele);DOM.removeElement(settings.styles);settings=null}},setStatus(state){if(controller){controller.ui.textStatus.innerText=state}}}}();

1
app/Resources/Tracker/scripts.min/settings.js Normal file
View File

@@ -0,0 +1 @@
const CONSTANTS={AUTOSCROLL_ACTION_NOTHING:"optNothing",AUTOSCROLL_ACTION_PAUSE:"optPause",AUTOSCROLL_ACTION_SWITCH:"optSwitch"};let IS_FIRST_RUN=false;const SETTINGS=function(){const settingsChangedEvents=[];const saveSettings=function(){DOM.saveToCookie("DHT_SETTINGS",root,60*60*24*365*5)};const triggerSettingsChanged=function(property){for(const callback of settingsChangedEvents){callback(property)}saveSettings()};const defineTriggeringProperty=function(obj,property,value){const name="_"+property;Object.defineProperty(obj,property,{get:()=>obj[name],set:value=>{obj[name]=value;triggerSettingsChanged(property)}});obj[name]=value};let loaded=DOM.loadFromCookie("DHT_SETTINGS");if(!loaded){loaded={_autoscroll:true,_afterFirstMsg:CONSTANTS.AUTOSCROLL_ACTION_PAUSE,_afterSavedMsg:CONSTANTS.AUTOSCROLL_ACTION_PAUSE};IS_FIRST_RUN=true}const root={onSettingsChanged(callback){settingsChangedEvents.push(callback)}};defineTriggeringProperty(root,"autoscroll",loaded._autoscroll);defineTriggeringProperty(root,"afterFirstMsg",loaded._afterFirstMsg);defineTriggeringProperty(root,"afterSavedMsg",loaded._afterSavedMsg);if(IS_FIRST_RUN){saveSettings()}return root}();

1
app/Resources/Tracker/scripts.min/state.js Normal file
View File

@@ -0,0 +1 @@
const STATE=function(){let serverPort=-1;let serverToken="";const post=function(endpoint,json){const aborter=new AbortController;const timeout=window.setTimeout(()=>aborter.abort(),5e3);return new Promise(async(resolve,reject)=>{let r;try{r=await fetch("http://127.0.0.1:"+serverPort+endpoint,{method:"POST",headers:{"Content-Type":"application/json","X-DHT-Token":serverToken},credentials:"omit",redirect:"error",body:JSON.stringify(json),signal:aborter.signal})}catch(e){if(e.name==="AbortError"){reject({status:"DISCONNECTED"});return}else{reject({status:"ERROR",message:e.message});return}}finally{window.clearTimeout(timeout)}if(r.status===200){resolve(r);return}try{const message=await r.text();reject({status:"ERROR",message:message})}catch(e){reject({status:"ERROR",message:e.message})}})};const trackingStateChangedListeners=[];let isTracking=false;const addedChannels=new Set;const addedUsers=new Set;return{setup(port,token){serverPort=port;serverToken=token},onTrackingStateChanged(callback){trackingStateChangedListeners.push(callback);callback(isTracking)},isTracking(){return isTracking},setIsTracking(state){if(isTracking!==state){isTracking=state;if(isTracking){addedChannels.clear();addedUsers.clear()}for(const callback of trackingStateChangedListeners){callback(isTracking)}}},async addDiscordChannel(serverInfo,channelInfo){if(addedChannels.has(channelInfo.id)){return}const server={id:serverInfo.id,name:serverInfo.name,type:serverInfo.type};const channel={id:channelInfo.id,name:channelInfo.name};if("extra"in channelInfo){const extra=channelInfo.extra;if("parent"in extra){channel.parent=extra.parent}channel.position=extra.position;channel.topic=extra.topic;channel.nsfw=extra.nsfw}await post("/track-channel",{server:server,channel:channel});addedChannels.add(channelInfo.id)},async addDiscordMessages(channelId,discordMessageArray){discordMessageArray=discordMessageArray.filter(msg=>(msg.type===0||msg.type===19||msg.type===21)&&msg.state==="SENT");if(discordMessageArray.length===0){return false}const userInfo={};let hasNewUsers=false;for(const msg of discordMessageArray){const user=msg.author;if(!addedUsers.has(user.id)){const obj={id:user.id,name:user.username};if(user.avatar){obj.avatar=user.avatar}if(!user.bot){obj.discriminator=user.discriminator}userInfo[user.id]=obj;hasNewUsers=true}}if(hasNewUsers){await post("/track-users",Object.values(userInfo));for(const id of Object.keys(userInfo)){addedUsers.add(id)}}const response=await post("/track-messages",discordMessageArray.map(msg=>{const obj={id:msg.id,sender:msg.author.id,channel:msg.channel_id,text:msg.content,timestamp:msg.timestamp.toDate().getTime()};if(msg.editedTimestamp!==null){obj.editTimestamp=msg.editedTimestamp.toDate().getTime()}if(msg.messageReference!==null){obj.repliedToId=msg.messageReference.message_id}if(msg.attachments.length>0){obj.attachments=msg.attachments.map(attachment=>{const mapped={id:attachment.id,name:attachment.filename,size:attachment.size,url:attachment.url};if(attachment.content_type){mapped.type=attachment.content_type}return mapped})}if(msg.embeds.length>0){obj.embeds=msg.embeds.map(embed=>{const mapped={};for(const key of Object.keys(embed)){if(key==="id"){continue}if(key==="rawTitle"){mapped["title"]=embed[key]}else if(key==="rawDescription"){mapped["description"]=embed[key]}else{mapped[key]=embed[key]}}return JSON.stringify(mapped)})}if(msg.reactions.length>0){obj.reactions=msg.reactions.map(reaction=>{const emoji=reaction.emoji;const mapped={count:reaction.count};if(emoji.id){mapped.id=emoji.id}if(emoji.name){mapped.name=emoji.name}if(emoji.animated){mapped.isAnimated=emoji.animated}return mapped})}return obj}));const anyNewMessages=await response.text();return anyNewMessages==="1"}}}();

96
app/Resources/Tracker/scripts/discord.js
View File

@@ -67,7 +67,14 @@ class DISCORD {
}
const messages = this.getMessages();
const hasChanged = messages.some(message => !previousMessages.has(message.id)) || !this.hasMoreMessages();
let hasChanged = false;
for (const message of messages) {
if (!previousMessages.has(message.id)) {
hasChanged = true;
break;
}
}
if (!hasChanged) {
return;
@@ -110,20 +117,16 @@ class DISCORD {
const messages = [];
for (const ele of this.getMessageElements()) {
try {
const props = this.getMessageElementProps(ele);
if (props != null) {
messages.push(props.message);
}
} catch (e) {
console.error("[DHT] Error extracing message data, skipping it.", e, ele, DOM.tryGetReactProps(ele));
const props = this.getMessageElementProps(ele);
if (props != null) {
messages.push(props.message);
}
}
return messages;
} catch (e) {
console.error("[DHT] Error retrieving messages.", e);
console.error(e);
return [];
}
}
@@ -137,27 +140,16 @@ class DISCORD {
try {
let obj;
try {
for (const child of DOM.getReactProps(DOM.queryReactClass("chatContent")).children) {
if (child && child.props && child.props.channel) {
obj = child.props.channel;
break;
}
}
} catch (e) {
console.error("[DHT] Error retrieving selected channel from 'chatContent' element.", e);
for (const ele of this.getMessageElements()) {
const props = this.getMessageElementProps(ele);
for (const ele of this.getMessageElements()) {
const props = this.getMessageElementProps(ele);
if (props != null) {
obj = props.channel;
break;
}
if (props != null) {
obj = props.channel;
break;
}
}
if (!obj || typeof obj.id !== "string") {
if (!obj) {
return null;
}
@@ -197,7 +189,7 @@ class DISCORD {
else if (obj.guild_id) {
const server = {
"id": obj.guild_id,
"name": document.querySelector("nav header h1[class*='name-']").innerText,
"name": document.querySelector("nav header > h1").innerText,
"type": "SERVER"
};
@@ -223,7 +215,7 @@ class DISCORD {
return null;
}
} catch (e) {
console.error("[DHT] Error retrieving selected channel.", e);
console.error(e);
return null;
}
}
@@ -236,38 +228,42 @@ class DISCORD {
if (dms) {
const currentChannel = DOM.queryReactClass("selected", dms);
const currentChannelContainer = currentChannel && currentChannel.closest("[class*='channel-']");
const nextChannel = currentChannelContainer && currentChannelContainer.nextElementSibling;
const nextChannel = currentChannel && currentChannel.nextElementSibling;
if (!nextChannel || !nextChannel.getAttribute("class").includes("channel-")) {
if (!nextChannel || !nextChannel.getAttribute("class").includes("channel-") || !("href" in nextChannel) || !nextChannel.href.includes("/@me/")) {
return false;
}
const nextChannelLink = nextChannel.querySelector("a[href*='/@me/']");
if (!nextChannelLink) {
return false;
else {
nextChannel.click();
nextChannel.scrollIntoView(true);
return true;
}
nextChannelLink.click();
nextChannelLink.scrollIntoView(true);
return true;
}
else {
const channelListEle = document.getElementById("channels");
const channelIcons = [
/* normal */ "M5.88657 21C5.57547 21 5.3399 20.7189 5.39427 20.4126L6.00001 17H2.59511C2.28449 17 2.04905 16.7198 2.10259 16.4138L2.27759 15.4138C2.31946 15.1746 2.52722 15 2.77011 15H6.35001L7.41001 9H4.00511C3.69449 9 3.45905 8.71977 3.51259 8.41381L3.68759 7.41381C3.72946 7.17456 3.93722 7 4.18011 7H7.76001L8.39677 3.41262C8.43914 3.17391 8.64664 3 8.88907 3H9.87344C10.1845 3 10.4201 3.28107 10.3657 3.58738L9.76001 7H15.76L16.3968 3.41262C16.4391 3.17391 16.6466 3 16.8891 3H17.8734C18.1845 3 18.4201 3.28107 18.3657 3.58738L17.76 7H21.1649C21.4755 7 21.711 7.28023 21.6574 7.58619L21.4824 8.58619C21.4406 8.82544 21.2328 9 20.9899 9H17.41L16.35 15H19.7549C20.0655 15 20.301 15.2802 20.2474 15.5862L20.0724 16.5862C20.0306 16.8254 19.8228 17 19.5799 17H16L15.3632 20.5874C15.3209 20.8261 15.1134 21 14.8709 21H13.8866C13.5755 21 13.3399 20.7189 13.3943 20.4126L14 17H8.00001L7.36325 20.5874C7.32088 20.8261 7.11337 21 6.87094 21H5.88657ZM9.41045 9L8.35045 15H14.3504L15.4104 9H9.41045Z",
/* normal + thread */ "M5.43309 21C5.35842 21 5.30189 20.9325 5.31494 20.859L5.99991 17H2.14274C2.06819 17 2.01168 16.9327 2.02453 16.8593L2.33253 15.0993C2.34258 15.0419 2.39244 15 2.45074 15H6.34991L7.40991 9H3.55274C3.47819 9 3.42168 8.93274 3.43453 8.85931L3.74253 7.09931C3.75258 7.04189 3.80244 7 3.86074 7H7.75991L8.45234 3.09903C8.46251 3.04174 8.51231 3 8.57049 3H10.3267C10.4014 3 10.4579 3.06746 10.4449 3.14097L9.75991 7H15.7599L16.4523 3.09903C16.4625 3.04174 16.5123 3 16.5705 3H18.3267C18.4014 3 18.4579 3.06746 18.4449 3.14097L17.7599 7H21.6171C21.6916 7 21.7481 7.06725 21.7353 7.14069L21.4273 8.90069C21.4172 8.95811 21.3674 9 21.3091 9H17.4099L17.0495 11.04H15.05L15.4104 9H9.41035L8.35035 15H10.5599V17H7.99991L7.30749 20.901C7.29732 20.9583 7.24752 21 7.18934 21H5.43309Z",
/* nsfw or private */ "M14 8C14 7.44772 13.5523 7 13 7H9.76001L10.3657 3.58738C10.4201 3.28107 10.1845 3 9.87344 3H8.88907C8.64664 3 8.43914 3.17391 8.39677 3.41262L7.76001 7H4.18011C3.93722 7 3.72946 7.17456 3.68759 7.41381L3.51259 8.41381C3.45905 8.71977 3.69449 9 4.00511 9H7.41001L6.35001 15H2.77011C2.52722 15 2.31946 15.1746 2.27759 15.4138L2.10259 16.4138C2.04905 16.7198 2.28449 17 2.59511 17H6.00001L5.39427 20.4126C5.3399 20.7189 5.57547 21 5.88657 21H6.87094C7.11337 21 7.32088 20.8261 7.36325 20.5874L8.00001 17H14L13.3943 20.4126C13.3399 20.7189 13.5755 21 13.8866 21H14.8709C15.1134 21 15.3209 20.8261 15.3632 20.5874L16 17H19.5799C19.8228 17 20.0306 16.8254 20.0724 16.5862L20.2474 15.5862C20.301 15.2802 20.0655 15 19.7549 15H16.35L16.6758 13.1558C16.7823 12.5529 16.3186 12 15.7063 12C15.2286 12 14.8199 12.3429 14.7368 12.8133L14.3504 15H8.35045L9.41045 9H13C13.5523 9 14 8.55228 14 8Z",
/* nsfw + thread */ "M14.4 7C14.5326 7 14.64 7.10745 14.64 7.24V8.76C14.64 8.89255 14.5326 9 14.4 9H9.41045L8.35045 15H10.56V17H8.00001L7.36325 20.5874C7.32088 20.8261 7.11337 21 6.87094 21H5.88657C5.57547 21 5.3399 20.7189 5.39427 20.4126L6.00001 17H2.59511C2.28449 17 2.04905 16.7198 2.10259 16.4138L2.27759 15.4138C2.31946 15.1746 2.52722 15 2.77011 15H6.35001L7.41001 9H4.00511C3.69449 9 3.45905 8.71977 3.51259 8.41381L3.68759 7.41381C3.72946 7.17456 3.93722 7 4.18011 7H7.76001L8.39677 3.41262C8.43914 3.17391 8.64664 3 8.88907 3H9.87344C10.1845 3 10.4201 3.28107 10.3657 3.58738L9.76001 7H14.4Z",
/* private + thread */ "M15.44 6.99992C15.5725 6.99992 15.68 7.10737 15.68 7.23992V8.75992C15.68 8.89247 15.5725 8.99992 15.44 8.99992H9.41045L8.35045 14.9999H10.56V16.9999H8.00001L7.36325 20.5873C7.32088 20.826 7.11337 20.9999 6.87094 20.9999H5.88657C5.57547 20.9999 5.3399 20.7189 5.39427 20.4125L6.00001 16.9999H2.59511C2.28449 16.9999 2.04905 16.7197 2.10259 16.4137L2.27759 15.4137C2.31946 15.1745 2.52722 14.9999 2.77011 14.9999H6.35001L7.41001 8.99992H4.00511C3.69449 8.99992 3.45905 8.71969 3.51259 8.41373L3.68759 7.41373C3.72946 7.17448 3.93722 6.99992 4.18011 6.99992H7.76001L8.39677 3.41254C8.43914 3.17384 8.64664 2.99992 8.88907 2.99992H9.87344C10.1845 2.99992 10.4201 3.28099 10.3657 3.58731L9.76001 6.99992H15.44Z"
];
const isValidChannelClass = cls => cls.includes("wrapper-") && !cls.includes("clickable-");
const isValidChannelType = ele => channelIcons.some(icon => !!ele.querySelector("path[d=\"" + icon + "\"]"));
const isValidChannel = ele => ele.childElementCount > 0 && isValidChannelClass(ele.children[0].className) && isValidChannelType(ele);
const channelListEle = document.querySelector("div[class*='sidebar'] > nav[class*='container'] > div[class*='scroller']");
if (!channelListEle) {
return false;
}
function getLinkElement(channel) {
return channel.querySelector("a[href^='/channels/'][role='link']");
}
const allTextChannels = Array.prototype.filter.call(channelListEle.querySelectorAll("[class*='containerDefault']"), ele => getLinkElement(ele) !== null);
const allChannels = Array.prototype.filter.call(channelListEle.querySelectorAll("[class*='containerDefault']"), isValidChannel);
let nextChannel = null;
for (let index = 0; index < allTextChannels.length - 1; index++) {
if (allTextChannels[index].className.includes("selected-")) {
nextChannel = allTextChannels[index + 1];
for (let index = 0; index < allChannels.length - 1; index++) {
if (allChannels[index].children[0].className.includes("modeSelected")) {
nextChannel = allChannels[index + 1];
break;
}
}
@@ -276,7 +272,7 @@ class DISCORD {
return false;
}
const nextChannelLink = getLinkElement(nextChannel);
const nextChannelLink = nextChannel.querySelector("a[href^='/channels/']");
if (!nextChannelLink) {
return false;
}

11
app/Resources/Tracker/scripts/dom.js
View File

@@ -71,15 +71,4 @@ class DOM {
key = keys.find(key => key.startsWith("__reactProps$"));
return key ? ele[key] : null;
}
/**
* Returns internal React state object of an element, or null if the retrieval throws.
*/
static tryGetReactProps(ele) {
try {
return this.getReactProps(ele);
} catch (e) {
return null;
}
}
}

1
app/Resources/Tracker/scripts/state.js
View File

@@ -304,6 +304,7 @@ const STATE = (function() {
});
}
obj["raw"] = JSON.stringify(msg);
return obj;
}));

7
app/Resources/Tracker/styles/controller.css
View File

@@ -1,18 +1,13 @@
#app-mount div[class*="app-"] {
#app-mount > div[class*="app-"] {
margin-bottom: 48px !important;
}
#app-mount div[class*="app-"] > div[class*="app-"] {
margin-bottom: 0 !important;
}
#dht-ctrl {
position: absolute;
bottom: 0;
width: 100%;
height: 48px;
background-color: #fff;
z-index: 1000000;
}
#dht-ctrl button {

4
app/Resources/Tracker/styles/settings.css
View File

@@ -7,7 +7,7 @@
background-color: #000;
opacity: 0.5;
display: block;
z-index: 1000001;
z-index: 1000;
}
#dht-cfg {
@@ -20,7 +20,7 @@
margin-top: -131px;
padding: 8px;
background-color: #fff;
z-index: 1000002;
z-index: 1001;
}
#dht-cfg-note {

42
app/Resources/Viewer/scripts/discord.js
View File

@@ -1,7 +1,7 @@
const DISCORD = (function() {
const regex = {
formatBold: /\*\*([\s\S]+?)\*\*(?!\*)/g,
formatItalic: /(.)?([_*])([\s\S]+?)\2(?!\2)/g,
formatItalic: /(.)?\*([\s\S]+?)\*(?!\*)/g,
formatUnderline: /__([\s\S]+?)__(?!_)/g,
formatStrike: /~~([\s\S]+?)~~(?!~)/g,
formatCodeInline: /(`+)\s*([\s\S]*?[^`])\s*\1(?!`)/g,
@@ -9,7 +9,7 @@ const DISCORD = (function() {
formatUrl: /(\b(?:https?|ftp|file):\/\/[-A-Z0-9+&@#\/%?=~_|!:,.;]*[-A-Z0-9+&@#\/%=~_|])/ig,
formatUrlNoEmbed: /<(\b(?:https?|ftp|file):\/\/[-A-Z0-9+&@#\/%?=~_|!:,.;]*[-A-Z0-9+&@#\/%=~_|])>/ig,
specialEscapedBacktick: /\\`/g,
specialEscapedSingle: /\\([*_\\])/g,
specialEscapedSingle: /\\([*\\])/g,
specialEscapedDouble: /\\__|_\\_|\\_\\_|\\~~|~\\~|\\~\\~/g,
specialUnescaped: /([*_~\\])/g,
mentionRole: /&lt;@&(\d+?)&gt;/g,
@@ -26,7 +26,6 @@ const DISCORD = (function() {
let templateUserAvatar;
let templateAttachmentDownload;
let templateEmbedImage;
let templateEmbedImageWithSize;
let templateEmbedRich;
let templateEmbedRichNoDescription;
let templateEmbedUrl;
@@ -47,8 +46,8 @@ const DISCORD = (function() {
.replace(regex.specialEscapedSingle, escapeHtmlMatch)
.replace(regex.specialEscapedDouble, full => full.replace(/\\/g, "").replace(/(.)/g, escapeHtmlMatch))
.replace(regex.formatBold, "<b>$1</b>")
.replace(regex.formatItalic, (full, pre, match) => pre === "\\" ? full : (pre || "") + "<i>" + match + "</i>")
.replace(regex.formatUnderline, "<u>$1</u>")
.replace(regex.formatItalic, (full, pre, char, match) => pre === "\\" ? full : (pre || "") + "<i>" + match + "</i>")
.replace(regex.formatStrike, "<s>$1</s>");
}
@@ -65,19 +64,6 @@ const DISCORD = (function() {
return "<p>" + processed + "</p>";
};
const getImageEmbed = function(url, image) {
if (!SETTINGS.enableImagePreviews) {
return "";
}
if (image.width && image.height) {
return templateEmbedImageWithSize.apply({ url, src: image.url, width: image.width, height: image.height });
}
else {
return templateEmbedImage.apply({ url, src: image.url });
}
};
return {
setup() {
templateChannelServer = new TEMPLATE([
@@ -128,12 +114,7 @@ const DISCORD = (function() {
// noinspection HtmlUnknownTarget
templateEmbedImage = new TEMPLATE([
"<a href='{url}' class='embed thumbnail loading'><img src='{src}' alt='' onload='DISCORD.handleImageLoad(this)' onerror='DISCORD.handleImageLoadError(this)'></a><br>"
].join(""));
// noinspection HtmlUnknownTarget
templateEmbedImageWithSize = new TEMPLATE([
"<a href='{url}' class='embed thumbnail loading'><img src='{src}' width='{width}' height='{height}' alt='' onload='DISCORD.handleImageLoad(this)' onerror='DISCORD.handleImageLoadError(this)'></a><br>"
"<a href='{url}' class='embed thumbnail'><img src='{src}' alt='(image attachment not found)'></a><br>"
].join(""));
// noinspection HtmlUnknownTarget
@@ -164,17 +145,6 @@ const DISCORD = (function() {
].join(""));
},
handleImageLoad(ele) {
ele.parentElement.classList.remove("loading");
},
handleImageLoadError(ele) {
// noinspection JSUnusedGlobalSymbols
ele.onerror = null;
ele.parentElement.classList.remove("loading");
ele.setAttribute("alt", "(image attachment not found)");
},
isImageAttachment(attachment) {
const dot = attachment.url.lastIndexOf(".");
const ext = dot === -1 ? "" : attachment.url.substring(dot).toLowerCase();
@@ -213,10 +183,10 @@ const DISCORD = (function() {
return templateEmbedUnsupported.apply(embed);
}
else if ("image" in embed && embed.image.url) {
return getImageEmbed(embed.url, embed.image);
return SETTINGS.enableImagePreviews ? templateEmbedImage.apply({ url: embed.url, src: embed.image.url }) : "";
}
else if ("thumbnail" in embed && embed.thumbnail.url) {
return getImageEmbed(embed.url, embed.thumbnail);
return SETTINGS.enableImagePreviews ? templateEmbedImage.apply({ url: embed.url, src: embed.thumbnail.url }) : "";
}
else if ("title" in embed && "description" in embed) {
return templateEmbedRich.apply(embed);

14
app/Resources/Viewer/styles/messages.css
View File

@@ -107,25 +107,11 @@
}
.message .thumbnail {
position: relative;
max-width: calc(100% - 20px);
max-height: 320px;
}
.message .thumbnail.loading::after {
content: "";
background: rgba(0, 0, 0, 0.75)
url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 300 300' preserveAspectRatio='xMidYMid'%3E %3Ccircle cx='150' cy='150' fill='none' stroke='%237983f5' stroke-width='8' r='42' stroke-dasharray='198 68'%3E %3CanimateTransform attributeName='transform' type='rotate' repeatCount='indefinite' dur='1.25s' values='0 150 150;360 150 150' keyTimes='0;1' /%3E %3C/circle%3E %3C/svg%3E")
no-repeat center center;
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
}
.message .thumbnail img {
width: auto;
max-width: 100%;
max-height: 320px;
border-radius: 3px;

27
app/Resources/minify.py
View File

@@ -1,27 +0,0 @@
#!/usr/bin/env python3
import glob
import os
import shutil
import sys
if os.name == "nt":
uglifyjs = os.path.abspath("../../lib/uglifyjs.cmd")
else:
uglifyjs = "uglifyjs"
if shutil.which(uglifyjs) is None:
print("Cannot find executable: {0}".format(uglifyjs))
sys.exit(1)
input_dir = os.path.abspath("./Tracker/scripts")
output_dir = os.path.abspath("../Desktop/bin/.res/scripts")
os.makedirs(output_dir, exist_ok=True)
for file in glob.glob(input_dir + "/*.js"):
name = os.path.basename(file)
print("Minifying {0}...".format(name))
os.system("{0} {1} -o {2}/{3}".format(uglifyjs, file, output_dir, name))
print("Done!")

15
app/Server/Data/Aggregations/DownloadStatusStatistics.cs
View File

@@ -1,15 +0,0 @@
namespace DHT.Server.Data.Aggregations {
public sealed class DownloadStatusStatistics {
public int EnqueuedCount { get; internal set; }
public ulong EnqueuedSize { get; internal set; }
public int SuccessfulCount { get; internal set; }
public ulong SuccessfulSize { get; internal set; }
public int FailedCount { get; internal set; }
public ulong FailedSize { get; internal set; }
public int SkippedCount { get; internal set; }
public ulong SkippedSize { get; internal set; }
}
}

14
app/Server/Data/Channel.cs
View File

@@ -1,11 +1,11 @@
namespace DHT.Server.Data {
public readonly struct Channel {
public ulong Id { get; init; }
public ulong Server { get; init; }
public string Name { get; init; }
public ulong? ParentId { get; init; }
public int? Position { get; init; }
public string? Topic { get; init; }
public bool? Nsfw { get; init; }
public ulong Id { get; internal init; }
public ulong Server { get; internal init; }
public string Name { get; internal init; }
public ulong? ParentId { get; internal init; }
public int? Position { get; internal init; }
public string? Topic { get; internal init; }
public bool? Nsfw { get; internal init; }
}
}

30
app/Server/Data/Download.cs
View File

@@ -1,30 +0,0 @@
using System;
using System.Net;
namespace DHT.Server.Data {
public readonly struct Download {
internal static Download NewSuccess(string url, byte[] data) {
return new Download(url, DownloadStatus.Success, (ulong) Math.Max(data.LongLength, 0), data);
}
internal static Download NewFailure(string url, HttpStatusCode? statusCode, ulong size) {
return new Download(url, statusCode.HasValue ? (DownloadStatus) (int) statusCode : DownloadStatus.GenericError, size);
}
public string Url { get; }
public DownloadStatus Status { get; }
public ulong Size { get; }
public byte[]? Data { get; }
internal Download(string url, DownloadStatus status, ulong size, byte[]? data = null) {
Url = url;
Status = status;
Size = size;
Data = data;
}
internal Download WithData(byte[] data) {
return new Download(Url, Status, Size, data);
}
}
}

12
app/Server/Data/DownloadStatus.cs
View File

@@ -1,12 +0,0 @@
using System.Net;
namespace DHT.Server.Data {
/// <summary>
/// Extends <see cref="HttpStatusCode"/> with custom status codes in the range 0-99.
/// </summary>
public enum DownloadStatus {
Enqueued = 0,
GenericError = 1,
Success = HttpStatusCode.OK
}
}

15
app/Server/Data/Filters/AttachmentFilter.cs
View File

@@ -1,15 +0,0 @@
namespace DHT.Server.Data.Filters {
public sealed class AttachmentFilter {
public ulong? MaxBytes { get; set; } = null;
public DownloadItemRules? DownloadItemRule { get; set; } = null;
public bool IsEmpty => MaxBytes == null &&
DownloadItemRule == null;
public enum DownloadItemRules {
OnlyNotPresent,
OnlyPresent
}
}
}

10
app/Server/Data/Filters/DownloadItemFilter.cs
View File

@@ -1,10 +0,0 @@
using System.Collections.Generic;
namespace DHT.Server.Data.Filters {
public sealed class DownloadItemFilter {
public HashSet<DownloadStatus>? IncludeStatuses { get; set; } = null;
public HashSet<DownloadStatus>? ExcludeStatuses { get; set; } = null;
public bool IsEmpty => IncludeStatuses == null && ExcludeStatuses == null;
}
}

10
app/Server/Data/Filters/MessageFilter.cs
View File

@@ -3,17 +3,11 @@ using System.Collections.Generic;
namespace DHT.Server.Data.Filters {
public sealed class MessageFilter {
public DateTime? StartDate { get; set; } = null;
public DateTime? EndDate { get; set; } = null;
public DateTime? StartDate { get; set; }
public DateTime? EndDate { get; set; }
public HashSet<ulong>? ChannelIds { get; set; } = null;
public HashSet<ulong>? UserIds { get; set; } = null;
public HashSet<ulong>? MessageIds { get; set; } = null;
public bool IsEmpty => StartDate == null &&
EndDate == null &&
ChannelIds == null &&
UserIds == null &&
MessageIds == null;
}
}

2
app/Server/Data/Filters/FilterRemovalMode.cs → app/Server/Data/Filters/MessageFilterRemovalMode.cs
View File

@@ -1,5 +1,5 @@
namespace DHT.Server.Data.Filters {
public enum FilterRemovalMode {
public enum MessageFilterRemovalMode {
KeepMatching,
RemoveMatching
}

21
app/Server/Data/Message.cs
View File

@@ -2,15 +2,16 @@ using System.Collections.Immutable;
namespace DHT.Server.Data {
public readonly struct Message {
public ulong Id { get; init; }
public ulong Sender { get; init; }
public ulong Channel { get; init; }
public string Text { get; init; }
public long Timestamp { get; init; }
public long? EditTimestamp { get; init; }
public ulong? RepliedToId { get; init; }
public ImmutableArray<Attachment> Attachments { get; init; }
public ImmutableArray<Embed> Embeds { get; init; }
public ImmutableArray<Reaction> Reactions { get; init; }
public ulong Id { get; internal init; }
public ulong Sender { get; internal init; }
public ulong Channel { get; internal init; }
public string Text { get; internal init; }
public long Timestamp { get; internal init; }
public long? EditTimestamp { get; internal init; }
public ulong? RepliedToId { get; internal init; }
public ImmutableArray<Attachment> Attachments { get; internal init; }
public ImmutableArray<Embed> Embeds { get; internal init; }
public ImmutableArray<Reaction> Reactions { get; internal init; }
public string? RawJson { get; internal init; }
}
}

6
app/Server/Data/Server.cs
View File

@@ -1,7 +1,7 @@
namespace DHT.Server.Data {
public readonly struct Server {
public ulong Id { get; init; }
public string Name { get; init; }
public ServerType? Type { get; init; }
public ulong Id { get; internal init; }
public string Name { get; internal init; }
public ServerType? Type { get; internal init; }
}
}

9
app/Server/Data/ServerType.cs
View File

@@ -24,15 +24,6 @@ namespace DHT.Server.Data {
};
}
public static string ToNiceString(ServerType? type) {
return type switch {
ServerType.Server => "Server",
ServerType.Group => "Group",
ServerType.DirectMessage => "DM",
_ => "Unknown"
};
}
internal static string ToJsonViewerString(ServerType? type) {
return type switch {
ServerType.Server => "server",

8
app/Server/Data/User.cs
View File

@@ -1,8 +1,8 @@
namespace DHT.Server.Data {
public readonly struct User {
public ulong Id { get; init; }
public string Name { get; init; }
public string? AvatarUrl { get; init; }
public string? Discriminator { get; init; }
public ulong Id { get; internal init; }
public string Name { get; internal init; }
public string? AvatarUrl { get; internal init; }
public string? Discriminator { get; internal init; }
}
}

23
app/Server/Database/DatabaseExtensions.cs
View File

@@ -1,29 +1,16 @@
using System.Collections.Generic;
using DHT.Server.Data;
namespace DHT.Server.Database {
public static class DatabaseExtensions {
public static void AddFrom(this IDatabaseFile target, IDatabaseFile source) {
target.AddServers(source.GetAllServers());
target.AddChannels(source.GetAllChannels());
target.AddUsers(source.GetAllUsers().ToArray());
target.AddMessages(source.GetMessages().ToArray());
foreach (var download in source.GetDownloadsWithoutData()) {
target.AddDownload(download.Status == DownloadStatus.Success ? source.GetDownloadWithData(download) : download);
}
}
internal static void AddServers(this IDatabaseFile target, IEnumerable<Data.Server> servers) {
foreach (var server in servers) {
foreach (var server in source.GetAllServers()) {
target.AddServer(server);
}
}
internal static void AddChannels(this IDatabaseFile target, IEnumerable<Channel> channels) {
foreach (var channel in channels) {
foreach (var channel in source.GetAllChannels()) {
target.AddChannel(channel);
}
target.AddUsers(source.GetAllUsers().ToArray());
target.AddMessages(source.GetMessages().ToArray());
}
}
}

41
app/Server/Database/DatabaseStatistics.cs
View File

@@ -1,46 +1,47 @@
using DHT.Utils.Models;
using System.ComponentModel;
using System.Runtime.CompilerServices;
namespace DHT.Server.Database {
/// <summary>
/// A live view of database statistics.
/// Some of the totals are computed asynchronously and may not reflect the most recent version of the database, or may not be available at all until computed for the first time.
/// </summary>
public sealed class DatabaseStatistics : BaseModel {
public sealed class DatabaseStatistics : INotifyPropertyChanged {
private long totalServers;
private long totalChannels;
private long totalUsers;
private long? totalMessages;
private long? totalAttachments;
private long? totalDownloads;
private long totalMessages;
public long TotalServers {
get => totalServers;
internal set => Change(ref totalServers, value);
internal set => Change(out totalServers, value);
}
public long TotalChannels {
get => totalChannels;
internal set => Change(ref totalChannels, value);
internal set => Change(out totalChannels, value);
}
public long TotalUsers {
get => totalUsers;
internal set => Change(ref totalUsers, value);
internal set => Change(out totalUsers, value);
}
public long? TotalMessages {
public long TotalMessages {
get => totalMessages;
internal set => Change(ref totalMessages, value);
internal set => Change(out totalMessages, value);
}
public long? TotalAttachments {
get => totalAttachments;
internal set => Change(ref totalAttachments, value);
public event PropertyChangedEventHandler? PropertyChanged;
private void Change<T>(out T field, T value, [CallerMemberName] string? propertyName = null) {
field = value;
PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(propertyName));
}
public long? TotalDownloads {
get => totalDownloads;
internal set => Change(ref totalDownloads, value);
public DatabaseStatistics Clone() {
return new DatabaseStatistics {
totalServers = totalServers,
totalChannels = totalChannels,
totalUsers = TotalUsers,
totalMessages = totalMessages
};
}
}
}

11
app/Server/Database/DatabaseStatisticsSnapshot.cs
View File

@@ -1,11 +0,0 @@
namespace DHT.Server.Database {
/// <summary>
/// A complete snapshot of database statistics at a particular point in time.
/// </summary>
public readonly struct DatabaseStatisticsSnapshot {
public long TotalServers { get; internal init; }
public long TotalChannels { get; internal init; }
public long TotalUsers { get; internal init; }
public long TotalMessages { get; internal init; }
}
}

40
app/Server/Database/DummyDatabaseFile.cs
View File

@@ -1,8 +1,6 @@
using System.Collections.Generic;
using DHT.Server.Data;
using DHT.Server.Data.Aggregations;
using DHT.Server.Data.Filters;
using DHT.Server.Download;
namespace DHT.Server.Database {
public sealed class DummyDatabaseFile : IDatabaseFile {
@@ -13,10 +11,6 @@ namespace DHT.Server.Database {
private DummyDatabaseFile() {}
public DatabaseStatisticsSnapshot SnapshotStatistics() {
return new();
}
public void AddServer(Data.Server server) {}
public List<Data.Server> GetAllServers() {
@@ -45,39 +39,7 @@ namespace DHT.Server.Database {
return new();
}
public HashSet<ulong> GetMessageIds(MessageFilter? filter = null) {
return new();
}
public void RemoveMessages(MessageFilter filter, FilterRemovalMode mode) {}
public int CountAttachments(AttachmentFilter? filter = null) {
return new();
}
public List<Data.Download> GetDownloadsWithoutData() {
return new();
}
public Data.Download GetDownloadWithData(Data.Download download) {
return download;
}
public void AddDownload(Data.Download download) {}
public void EnqueueDownloadItems(AttachmentFilter? filter = null) {}
public List<DownloadItem> GetEnqueuedDownloadItems(int count) {
return new();
}
public void RemoveDownloadItems(DownloadItemFilter? filter, FilterRemovalMode mode) {}
public DownloadStatusStatistics GetDownloadStatusStatistics() {
return new();
}
public void Vacuum() {}
public void RemoveMessages(MessageFilter filter, MessageFilterRemovalMode mode) {}
public void Dispose() {}
}

112
app/Server/Database/Export/ViewerJsonExport.cs
View File

@@ -1,19 +1,13 @@
using System.Collections.Generic;
using System.IO;
using System.Dynamic;
using System.Linq;
using System.Text.Json;
using System.Threading.Tasks;
using DHT.Server.Data;
using DHT.Server.Data.Filters;
using DHT.Utils.Logging;
namespace DHT.Server.Database.Export {
public static class ViewerJsonExport {
private static readonly Log Log = Log.ForType(typeof(ViewerJsonExport));
public static async Task Generate(Stream stream, IDatabaseFile db, MessageFilter? filter = null) {
var perf = Log.Start();
public static string Generate(IDatabaseFile db, MessageFilter? filter = null) {
var includedUserIds = new HashSet<ulong>();
var includedChannelIds = new HashSet<ulong>();
var includedServerIds = new HashSet<ulong>();
@@ -33,32 +27,23 @@ namespace DHT.Server.Database.Export {
}
}
var opts = new JsonSerializerOptions();
opts.Converters.Add(new ViewerJsonSnowflakeSerializer());
var users = GenerateUserList(db, includedUserIds, out var userindex, out var userIndices);
var servers = GenerateServerList(db, includedServerIds, out var serverindex);
var channels = GenerateChannelList(includedChannels, serverindex);
perf.Step("Collect database data");
var value = new {
return JsonSerializer.Serialize(new {
meta = new { users, userindex, servers, channels },
data = GenerateMessageList(includedMessages, userIndices)
};
perf.Step("Generate value object");
var opts = new JsonSerializerOptions();
opts.Converters.Add(new ViewerJsonSnowflakeSerializer());
await JsonSerializer.SerializeAsync(stream, value, opts);
perf.Step("Serialize to JSON");
perf.End();
}, opts);
}
private static object GenerateUserList(IDatabaseFile db, HashSet<ulong> userIds, out List<string> userindex, out Dictionary<ulong, object> userIndices) {
var users = new Dictionary<string, object>();
private static dynamic GenerateUserList(IDatabaseFile db, HashSet<ulong> userIds, out List<string> userindex, out Dictionary<ulong, int> userIndices) {
var users = new Dictionary<string, dynamic>();
userindex = new List<string>();
userIndices = new Dictionary<ulong, object>();
userIndices = new Dictionary<ulong, int>();
foreach (var user in db.GetAllUsers()) {
var id = user.Id;
@@ -66,16 +51,15 @@ namespace DHT.Server.Database.Export {
continue;
}
var obj = new Dictionary<string, object> {
["name"] = user.Name
};
dynamic obj = new ExpandoObject();
obj.name = user.Name;
if (user.AvatarUrl != null) {
obj["avatar"] = user.AvatarUrl;
obj.avatar = user.AvatarUrl;
}
if (user.Discriminator != null) {
obj["tag"] = user.Discriminator;
obj.tag = user.Discriminator;
}
var idStr = id.ToString();
@@ -87,9 +71,9 @@ namespace DHT.Server.Database.Export {
return users;
}
private static object GenerateServerList(IDatabaseFile db, HashSet<ulong> serverIds, out Dictionary<ulong, object> serverIndices) {
var servers = new List<object>();
serverIndices = new Dictionary<ulong, object>();
private static dynamic GenerateServerList(IDatabaseFile db, HashSet<ulong> serverIds, out Dictionary<ulong, int> serverIndices) {
var servers = new List<dynamic>();
serverIndices = new Dictionary<ulong, int>();
foreach (var server in db.GetAllServers()) {
var id = server.Id;
@@ -98,38 +82,37 @@ namespace DHT.Server.Database.Export {
}
serverIndices[id] = servers.Count;
servers.Add(new Dictionary<string, object> {
["name"] = server.Name,
["type"] = ServerTypes.ToJsonViewerString(server.Type)
servers.Add(new {
name = server.Name,
type = ServerTypes.ToJsonViewerString(server.Type)
});
}
return servers;
}
private static object GenerateChannelList(List<Channel> includedChannels, Dictionary<ulong, object> serverIndices) {
var channels = new Dictionary<string, object>();
private static dynamic GenerateChannelList(List<Channel> includedChannels, Dictionary<ulong, int> serverIndices) {
var channels = new Dictionary<string, dynamic>();
foreach (var channel in includedChannels) {
var obj = new Dictionary<string, object> {
["server"] = serverIndices[channel.Server],
["name"] = channel.Name
};
dynamic obj = new ExpandoObject();
obj.server = serverIndices[channel.Server];
obj.name = channel.Name;
if (channel.ParentId != null) {
obj["parent"] = channel.ParentId;
obj.parent = channel.ParentId;
}
if (channel.Position != null) {
obj["position"] = channel.Position;
obj.position = channel.Position;
}
if (channel.Topic != null) {
obj["topic"] = channel.Topic;
obj.topic = channel.Topic;
}
if (channel.Nsfw != null) {
obj["nsfw"] = channel.Nsfw;
obj.nsfw = channel.Nsfw;
}
channels[channel.Id.ToString()] = obj;
@@ -138,55 +121,54 @@ namespace DHT.Server.Database.Export {
return channels;
}
private static object GenerateMessageList(List<Message> includedMessages, Dictionary<ulong, object> userIndices) {
var data = new Dictionary<string, Dictionary<string, object>>();
private static dynamic GenerateMessageList(List<Message> includedMessages, Dictionary<ulong, int> userIndices) {
var data = new Dictionary<string, Dictionary<string, dynamic>>();
foreach (var grouping in includedMessages.GroupBy(static message => message.Channel)) {
var channel = grouping.Key.ToString();
var channelData = new Dictionary<string, object>();
var channelData = new Dictionary<string, dynamic>();
foreach (var message in grouping) {
var obj = new Dictionary<string, object> {
["u"] = userIndices[message.Sender],
["t"] = message.Timestamp
};
dynamic obj = new ExpandoObject();
obj.u = userIndices[message.Sender];
obj.t = message.Timestamp;
if (!string.IsNullOrEmpty(message.Text)) {
obj["m"] = message.Text;
obj.m = message.Text;
}
if (message.EditTimestamp != null) {
obj["te"] = message.EditTimestamp;
obj.te = message.EditTimestamp;
}
if (message.RepliedToId != null) {
obj["r"] = message.RepliedToId.Value;
obj.r = message.RepliedToId.Value;
}
if (!message.Attachments.IsEmpty) {
obj["a"] = message.Attachments.Select(static attachment => new Dictionary<string, object> {
{ "url", attachment.Url }
obj.a = message.Attachments.Select(static attachment => new {
url = attachment.Url
}).ToArray();
}
if (!message.Embeds.IsEmpty) {
obj["e"] = message.Embeds.Select(static embed => embed.Json).ToArray();
obj.e = message.Embeds.Select(static embed => embed.Json).ToArray();
}
if (!message.Reactions.IsEmpty) {
obj["re"] = message.Reactions.Select(static reaction => {
var r = new Dictionary<string, object>();
obj.re = message.Reactions.Select(static reaction => {
dynamic r = new ExpandoObject();
if (reaction.EmojiId != null) {
r["id"] = reaction.EmojiId.Value;
r.id = reaction.EmojiId.Value;
}
if (reaction.EmojiName != null) {
r["n"] = reaction.EmojiName;
r.n = reaction.EmojiName;
}
r["a"] = reaction.EmojiFlags.HasFlag(EmojiFlags.Animated);
r["c"] = reaction.Count;
r.a = reaction.EmojiFlags.HasFlag(EmojiFlags.Animated);
r.c = reaction.Count;
return r;
});
}

19
app/Server/Database/IDatabaseFile.cs
View File

@@ -1,15 +1,12 @@
using System;
using System.Collections.Generic;
using DHT.Server.Data;
using DHT.Server.Data.Aggregations;
using DHT.Server.Data.Filters;
using DHT.Server.Download;
namespace DHT.Server.Database {
public interface IDatabaseFile : IDisposable {
string Path { get; }
DatabaseStatistics Statistics { get; }
DatabaseStatisticsSnapshot SnapshotStatistics();
void AddServer(Data.Server server);
List<Data.Server> GetAllServers();
@@ -23,20 +20,6 @@ namespace DHT.Server.Database {
void AddMessages(Message[] messages);
int CountMessages(MessageFilter? filter = null);
List<Message> GetMessages(MessageFilter? filter = null);
HashSet<ulong> GetMessageIds(MessageFilter? filter = null);
void RemoveMessages(MessageFilter filter, FilterRemovalMode mode);
int CountAttachments(AttachmentFilter? filter = null);
void AddDownload(Data.Download download);
List<Data.Download> GetDownloadsWithoutData();
Data.Download GetDownloadWithData(Data.Download download);
void EnqueueDownloadItems(AttachmentFilter? filter = null);
List<DownloadItem> GetEnqueuedDownloadItems(int count);
void RemoveDownloadItems(DownloadItemFilter? filter, FilterRemovalMode mode);
DownloadStatusStatistics GetDownloadStatusStatistics();
void Vacuum();
void RemoveMessages(MessageFilter filter, MessageFilterRemovalMode mode);
}
}

21
app/Server/Database/Import/FakeSnowflake.cs
View File

@@ -1,21 +0,0 @@
using System;
namespace DHT.Server.Database.Import {
/// <summary>
/// https://discord.com/developers/docs/reference#snowflakes
/// </summary>
public sealed class FakeSnowflake {
private const ulong DiscordEpoch = 1420070400000UL;
private ulong id;
public FakeSnowflake() {
var unixMillis = (ulong) (DateTime.UtcNow.Subtract(DateTime.UnixEpoch).Ticks / TimeSpan.TicksPerMillisecond);
this.id = (unixMillis - DiscordEpoch) << 22;
}
internal ulong Next() {
return id++;
}
}
}

263
app/Server/Database/Import/LegacyArchiveImport.cs
View File

@@ -1,263 +0,0 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Linq;
using System.Text.Json;
using System.Threading.Tasks;
using DHT.Server.Data;
using DHT.Utils.Collections;
using DHT.Utils.Http;
using DHT.Utils.Logging;
using Microsoft.AspNetCore.StaticFiles;
namespace DHT.Server.Database.Import {
public static class LegacyArchiveImport {
private static readonly Log Log = Log.ForType(typeof(LegacyArchiveImport));
private static readonly FileExtensionContentTypeProvider ContentTypeProvider = new ();
public static async Task<bool> Read(Stream stream, IDatabaseFile db, FakeSnowflake fakeSnowflake, Func<Data.Server[], Task<Dictionary<Data.Server, ulong>?>> askForServerIds) {
var perf = Log.Start();
var root = await JsonSerializer.DeserializeAsync<JsonElement>(stream);
try {
var meta = root.RequireObject("meta");
var data = root.RequireObject("data");
perf.Step("Deserialize JSON");
var users = ReadUserList(meta);
var servers = ReadServerList(meta, fakeSnowflake);
var newServersOnly = new HashSet<Data.Server>(servers);
var oldServersById = db.GetAllServers().ToDictionary(static server => server.Id, static server => server);
var oldChannels = db.GetAllChannels();
var oldChannelsById = oldChannels.ToDictionary(static channel => channel.Id, static channel => channel);
foreach (var (channelId, serverIndex) in ReadChannelToServerIndexMapping(meta, servers)) {
if (oldChannelsById.TryGetValue(channelId, out var oldChannel) && oldServersById.TryGetValue(oldChannel.Server, out var oldServer) && newServersOnly.Remove(servers[serverIndex])) {
servers[serverIndex] = oldServer;
}
}
perf.Step("Read server and user list");
if (newServersOnly.Count > 0) {
var askedServerIds = await askForServerIds(newServersOnly.ToArray());
if (askedServerIds == null) {
return false;
}
perf.Step("Ask for server IDs");
for (var i = 0; i < servers.Length; i++) {
var server = servers[i];
if (askedServerIds.TryGetValue(server, out var serverId)) {
servers[i] = new Data.Server {
Id = serverId,
Name = server.Name,
Type = server.Type
};
}
}
}
var channels = ReadChannelList(meta, servers);
perf.Step("Read channel list");
var oldMessageIds = db.GetMessageIds();
var newMessages = channels.SelectMany(channel => ReadMessages(data, channel, users, fakeSnowflake))
.Where(message => !oldMessageIds.Contains(message.Id))
.ToArray();
perf.Step("Read messages");
db.AddUsers(users);
db.AddServers(servers);
db.AddChannels(channels);
db.AddMessages(newMessages);
perf.Step("Import into database");
} catch (HttpException e) {
throw new JsonException(e.Message);
}
perf.End();
return true;
}
private static User[] ReadUserList(JsonElement meta) {
const string UsersPath = "meta.users[]";
static ulong ParseUserIndex(JsonElement element, int index) {
return ulong.Parse(element.GetString() ?? throw new JsonException("Expected key 'meta.userindex[" + index + "]' to be a string."));
}
var userindex = meta.RequireArray("userindex", "meta")
.Select(static (item, index) => (ParseUserIndex(item, index), index))
.ToDictionary();
var users = new User[userindex.Count];
foreach (var item in meta.RequireObject("users", "meta").EnumerateObject()) {
var path = UsersPath + "." + item.Name;
var userId = ulong.Parse(item.Name);
var userObj = item.Value;
users[userindex[userId]] = new User {
Id = userId,
Name = userObj.RequireString("name", path),
AvatarUrl = userObj.HasKey("avatar") ? userObj.RequireString("avatar", path) : null,
Discriminator = userObj.HasKey("tag") ? userObj.RequireString("tag", path) : null
};
}
return users;
}
private static Data.Server[] ReadServerList(JsonElement meta, FakeSnowflake fakeSnowflake) {
const string ServersPath = "meta.servers[]";
return meta.RequireArray("servers", "meta").Select(serverObj => new Data.Server {
Id = fakeSnowflake.Next(),
Name = serverObj.RequireString("name", ServersPath),
Type = ServerTypes.FromString(serverObj.RequireString("type", ServersPath))
}).ToArray();
}
private const string ChannelsPath = "meta.channels";
private static Dictionary<ulong, int> ReadChannelToServerIndexMapping(JsonElement meta, Data.Server[] servers) {
return meta.RequireObject("channels", "meta").EnumerateObject().Select(item => {
var path = ChannelsPath + "." + item.Name;
var channelId = ulong.Parse(item.Name);
var channelObj = item.Value;
return (channelId, channelObj.RequireInt("server", path, min: 0, max: servers.Length - 1));
}).ToDictionary();
}
private static Channel[] ReadChannelList(JsonElement meta, Data.Server[] servers) {
return meta.RequireObject("channels", "meta").EnumerateObject().Select(item => {
var path = ChannelsPath + "." + item.Name;
var channelId = ulong.Parse(item.Name);
var channelObj = item.Value;
return new Channel {
Id = channelId,
Server = servers[channelObj.RequireInt("server", path, min: 0, max: servers.Length - 1)].Id,
Name = channelObj.RequireString("name", path),
Position = channelObj.HasKey("position") ? channelObj.RequireInt("position", path, min: 0) : null,
Topic = channelObj.HasKey("topic") ? channelObj.RequireString("topic", path) : null,
Nsfw = channelObj.HasKey("nsfw") ? channelObj.RequireBool("nsfw", path) : null
};
}).ToArray();
}
private static Message[] ReadMessages(JsonElement data, Channel channel, User[] users, FakeSnowflake fakeSnowflake) {
const string DataPath = "data";
var channelId = channel.Id;
var channelIdStr = channelId.ToString();
var messagesObj = data.HasKey(channelIdStr) ? data.RequireObject(channelIdStr, DataPath) : (JsonElement?) null;
if (messagesObj == null) {
return Array.Empty<Message>();
}
return messagesObj.Value.EnumerateObject().Select(item => {
var path = DataPath + "." + item.Name;
var messageId = ulong.Parse(item.Name);
var messageObj = item.Value;
return new Message {
Id = messageId,
Sender = users[messageObj.RequireInt("u", path, min: 0, max: users.Length - 1)].Id,
Channel = channelId,
Text = messageObj.HasKey("m") ? messageObj.RequireString("m", path) : string.Empty,
Timestamp = messageObj.RequireLong("t", path),
EditTimestamp = messageObj.HasKey("te") ? messageObj.RequireLong("te", path) : null,
RepliedToId = messageObj.HasKey("r") ? messageObj.RequireSnowflake("r", path) : null,
Attachments = messageObj.HasKey("a") ? ReadMessageAttachments(messageObj.RequireArray("a", path), fakeSnowflake, path + ".a[]").ToImmutableArray() : ImmutableArray<Attachment>.Empty,
Embeds = messageObj.HasKey("e") ? ReadMessageEmbeds(messageObj.RequireArray("e", path), path + ".e[]").ToImmutableArray() : ImmutableArray<Embed>.Empty,
Reactions = messageObj.HasKey("re") ? ReadMessageReactions(messageObj.RequireArray("re", path), path + ".re[]").ToImmutableArray() : ImmutableArray<Reaction>.Empty
};
}).ToArray();
}
[SuppressMessage("ReSharper", "ConvertToLambdaExpression")]
private static IEnumerable<Attachment> ReadMessageAttachments(JsonElement.ArrayEnumerator attachmentsArray, FakeSnowflake fakeSnowflake, string path) {
return attachmentsArray.Select(attachmentObj => {
string url = attachmentObj.RequireString("url", path);
string name = url[(url.LastIndexOf('/') + 1)..];
string? type = ContentTypeProvider.TryGetContentType(name, out var contentType) ? contentType : null;
return new Attachment {
Id = fakeSnowflake.Next(),
Name = name,
Type = type,
Url = url,
Size = 0 // unknown size
};
}).DistinctByKeyStable(static attachment => {
// Some Discord messages have duplicate attachments with the same id for unknown reasons.
return attachment.Id;
});
}
private static IEnumerable<Embed> ReadMessageEmbeds(JsonElement.ArrayEnumerator embedsArray, string path) {
// Some rich embeds are missing URLs which causes a missing 'url' key.
return embedsArray.Where(static embedObj => embedObj.HasKey("url")).Select(embedObj => {
string url = embedObj.RequireString("url", path);
string type = embedObj.RequireString("type", path);
var embedJson = new Dictionary<string, object> {
{ "url", url },
{ "type", type },
{ "dht_legacy", true }
};
if (type == "image") {
embedJson["image"] = new Dictionary<string, string> {
{ "url", url }
};
}
else if (type == "rich") {
if (embedObj.HasKey("t")) {
embedJson["title"] = embedObj.RequireString("t", path);
}
if (embedObj.HasKey("d")) {
embedJson["description"] = embedObj.RequireString("d", path);
}
}
return new Embed {
Json = JsonSerializer.Serialize(embedJson)
};
});
}
private static IEnumerable<Reaction> ReadMessageReactions(JsonElement.ArrayEnumerator reactionsArray, string path) {
return reactionsArray.Select(reactionObj => {
var id = reactionObj.HasKey("id") ? reactionObj.RequireSnowflake("id", path) : (ulong?) null;
var name = reactionObj.HasKey("n") ? reactionObj.RequireString("n", path) : null;
if (id == null && name == null) {
throw new JsonException("Expected key '" + path + ".id' and/or '" + path + ".n' to be present.");
}
return new Reaction {
EmojiId = id,
EmojiName = name,
EmojiFlags = reactionObj.HasKey("an") && reactionObj.RequireBool("an", path) ? EmojiFlags.Animated : EmojiFlags.None,
Count = reactionObj.RequireInt("c", path, min: 0)
};
});
}
}
}

82
app/Server/Database/Sqlite/Schema.cs
View File

@@ -1,29 +1,32 @@
using System;
using System.Threading.Tasks;
using DHT.Server.Database.Exceptions;
using DHT.Server.Database.Sqlite.Utils;
using DHT.Utils.Logging;
using Microsoft.Data.Sqlite;
namespace DHT.Server.Database.Sqlite {
sealed class Schema {
internal const int Version = 4;
internal const int Version = 3;
private static readonly Log Log = Log.ForType<Schema>();
private readonly SqliteConnection conn;
private readonly ISqliteConnection conn;
public Schema(ISqliteConnection conn) {
public Schema(SqliteConnection conn) {
this.conn = conn;
}
private SqliteCommand Sql(string sql) {
var cmd = conn.CreateCommand();
cmd.CommandText = sql;
return cmd;
}
private void Execute(string sql) {
conn.Command(sql).ExecuteNonQuery();
Sql(sql).ExecuteNonQuery();
}
public async Task<bool> Setup(Func<Task<bool>> checkCanUpgradeSchemas) {
Execute(@"CREATE TABLE IF NOT EXISTS metadata (key TEXT PRIMARY KEY, value TEXT)");
var dbVersionStr = conn.SelectScalar("SELECT value FROM metadata WHERE key = 'version'");
var dbVersionStr = Sql("SELECT value FROM metadata WHERE key = 'version'").ExecuteScalar();
if (dbVersionStr == null) {
InitializeSchemas();
}
@@ -71,7 +74,9 @@ namespace DHT.Server.Database.Sqlite {
sender_id INTEGER NOT NULL,
channel_id INTEGER NOT NULL,
text TEXT NOT NULL,
timestamp INTEGER NOT NULL)");
timestamp INTEGER NOT NULL,
edit_timestamp INTEGER,
replied_to_id INTEGER)");
Execute(@"CREATE TABLE attachments (
message_id INTEGER NOT NULL,
@@ -92,9 +97,7 @@ namespace DHT.Server.Database.Sqlite {
emoji_flags INTEGER NOT NULL,
count INTEGER NOT NULL)");
CreateMessageEditTimestampTable();
CreateMessageRepliedToTable();
CreateDownloadsTable();
CreateMessagesRawTable();
Execute("CREATE INDEX attachments_message_ix ON attachments(message_id)");
Execute("CREATE INDEX embeds_message_ix ON embeds(message_id)");
@@ -103,63 +106,22 @@ namespace DHT.Server.Database.Sqlite {
Execute("INSERT INTO metadata (key, value) VALUES ('version', " + Version + ")");
}
private void CreateMessageEditTimestampTable() {
Execute(@"CREATE TABLE edit_timestamps (
message_id INTEGER PRIMARY KEY NOT NULL,
edit_timestamp INTEGER NOT NULL)");
}
private void CreateMessageRepliedToTable() {
Execute(@"CREATE TABLE replied_to (
message_id INTEGER PRIMARY KEY NOT NULL,
replied_to_id INTEGER NOT NULL)");
}
private void CreateDownloadsTable() {
Execute(@"CREATE TABLE downloads (
url TEXT NOT NULL PRIMARY KEY,
status INTEGER NOT NULL,
size INTEGER NOT NULL,
blob BLOB)");
}
private void UpgradeSchemas(int dbVersion) {
var perf = Log.Start("from version " + dbVersion);
Execute("UPDATE metadata SET value = " + Version + " WHERE key = 'version'");
if (dbVersion <= 1) {
Execute("ALTER TABLE channels ADD parent_id INTEGER");
perf.Step("Upgrade to version 2");
}
if (dbVersion <= 2) {
CreateMessageEditTimestampTable();
CreateMessageRepliedToTable();
Execute(@"INSERT INTO edit_timestamps (message_id, edit_timestamp)
SELECT message_id, edit_timestamp FROM messages
WHERE edit_timestamp IS NOT NULL");
Execute(@"INSERT INTO replied_to (message_id, replied_to_id)
SELECT message_id, replied_to_id FROM messages
WHERE replied_to_id IS NOT NULL");
Execute("ALTER TABLE messages DROP COLUMN replied_to_id");
Execute("ALTER TABLE messages DROP COLUMN edit_timestamp");
perf.Step("Upgrade to version 3");
Execute("VACUUM");
perf.Step("Vacuum");
CreateMessagesRawTable();
}
}
if (dbVersion <= 3) {
CreateDownloadsTable();
perf.Step("Upgrade to version 4");
}
perf.End();
private void CreateMessagesRawTable() {
Execute(@"CREATE TABLE messages_raw (
message_id INTEGER PRIMARY KEY NOT NULL,
json BLOB)");
}
}
}

624
app/Server/Database/Sqlite/SqliteDatabaseFile.cs
View File

@@ -4,156 +4,103 @@ using System.Collections.Immutable;
using System.Text;
using System.Threading.Tasks;
using DHT.Server.Data;
using DHT.Server.Data.Aggregations;
using DHT.Server.Data.Filters;
using DHT.Server.Database.Sqlite.Utils;
using DHT.Server.Download;
using DHT.Utils.Collections;
using DHT.Utils.Logging;
using DHT.Utils.Tasks;
using DHT.Utils.Compression;
using Microsoft.Data.Sqlite;
namespace DHT.Server.Database.Sqlite {
public sealed class SqliteDatabaseFile : IDatabaseFile {
private const int DefaultPoolSize = 5;
public static async Task<SqliteDatabaseFile?> OpenOrCreate(string path, Func<Task<bool>> checkCanUpgradeSchemas) {
var connectionString = new SqliteConnectionStringBuilder {
string connectionString = new SqliteConnectionStringBuilder {
DataSource = path,
Mode = SqliteOpenMode.ReadWriteCreate,
};
Mode = SqliteOpenMode.ReadWriteCreate
}.ToString();
var pool = new SqliteConnectionPool(connectionString, DefaultPoolSize);
bool wasOpened;
var conn = new SqliteConnection(connectionString);
conn.Open();
using (var conn = pool.Take()) {
wasOpened = await new Schema(conn).Setup(checkCanUpgradeSchemas);
}
if (wasOpened) {
return new SqliteDatabaseFile(path, pool);
}
else {
pool.Dispose();
return null;
}
return await new Schema(conn).Setup(checkCanUpgradeSchemas) ? new SqliteDatabaseFile(path, conn) : null;
}
public string Path { get; }
public DatabaseStatistics Statistics { get; }
private readonly Log log;
private readonly SqliteConnectionPool pool;
private readonly AsyncValueComputer<long>.Single totalMessagesComputer;
private readonly AsyncValueComputer<long>.Single totalAttachmentsComputer;
private readonly AsyncValueComputer<long>.Single totalDownloadsComputer;
private SqliteDatabaseFile(string path, SqliteConnectionPool pool) {
this.log = Log.ForType(typeof(SqliteDatabaseFile), System.IO.Path.GetFileName(path));
this.pool = pool;
this.totalMessagesComputer = AsyncValueComputer<long>.WithResultProcessor(UpdateMessageStatistics).WithOutdatedResults().BuildWithComputer(ComputeMessageStatistics);
this.totalAttachmentsComputer = AsyncValueComputer<long>.WithResultProcessor(UpdateAttachmentStatistics).WithOutdatedResults().BuildWithComputer(ComputeAttachmentStatistics);
this.totalDownloadsComputer = AsyncValueComputer<long>.WithResultProcessor(UpdateDownloadStatistics).WithOutdatedResults().BuildWithComputer(ComputeDownloadStatistics);
private readonly SqliteConnection conn;
private SqliteDatabaseFile(string path, SqliteConnection conn) {
this.conn = conn;
this.Path = path;
this.Statistics = new DatabaseStatistics();
using (var conn = pool.Take()) {
UpdateServerStatistics(conn);
UpdateChannelStatistics(conn);
UpdateUserStatistics(conn);
}
totalMessagesComputer.Recompute();
totalAttachmentsComputer.Recompute();
totalDownloadsComputer.Recompute();
UpdateServerStatistics();
UpdateChannelStatistics();
UpdateUserStatistics();
UpdateMessageStatistics();
}
public void Dispose() {
pool.Dispose();
}
public DatabaseStatisticsSnapshot SnapshotStatistics() {
return new DatabaseStatisticsSnapshot {
TotalServers = Statistics.TotalServers,
TotalChannels = Statistics.TotalChannels,
TotalUsers = Statistics.TotalUsers,
TotalMessages = ComputeMessageStatistics()
};
conn.Dispose();
}
public void AddServer(Data.Server server) {
using var conn = pool.Take();
using var cmd = conn.Upsert("servers", new[] {
("id", SqliteType.Integer),
("name", SqliteType.Text),
("type", SqliteType.Text)
"id", "name", "type"
});
cmd.Set(":id", server.Id);
cmd.Set(":name", server.Name);
cmd.Set(":type", ServerTypes.ToString(server.Type));
var serverParams = cmd.Parameters;
serverParams.AddAndSet(":id", server.Id);
serverParams.AddAndSet(":name", server.Name);
serverParams.AddAndSet(":type", ServerTypes.ToString(server.Type));
cmd.ExecuteNonQuery();
UpdateServerStatistics(conn);
UpdateServerStatistics();
}
public List<Data.Server> GetAllServers() {
var perf = log.Start();
var list = new List<Data.Server>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT id, name, type FROM servers");
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
list.Add(new Data.Server {
Id = reader.GetUint64(0),
Id = (ulong) reader.GetInt64(0),
Name = reader.GetString(1),
Type = ServerTypes.FromString(reader.GetString(2))
});
}
perf.End();
return list;
}
public void AddChannel(Channel channel) {
using var conn = pool.Take();
using var cmd = conn.Upsert("channels", new[] {
("id", SqliteType.Integer),
("server", SqliteType.Integer),
("name", SqliteType.Text),
("parent_id", SqliteType.Integer),
("position", SqliteType.Integer),
("topic", SqliteType.Text),
("nsfw", SqliteType.Integer)
"id", "server", "name", "parent_id", "position", "topic", "nsfw"
});
cmd.Set(":id", channel.Id);
cmd.Set(":server", channel.Server);
cmd.Set(":name", channel.Name);
cmd.Set(":parent_id", channel.ParentId);
cmd.Set(":position", channel.Position);
cmd.Set(":topic", channel.Topic);
cmd.Set(":nsfw", channel.Nsfw);
var channelParams = cmd.Parameters;
channelParams.AddAndSet(":id", channel.Id);
channelParams.AddAndSet(":server", channel.Server);
channelParams.AddAndSet(":name", channel.Name);
channelParams.AddAndSet(":parent_id", channel.ParentId);
channelParams.AddAndSet(":position", channel.Position);
channelParams.AddAndSet(":topic", channel.Topic);
channelParams.AddAndSet(":nsfw", channel.Nsfw);
cmd.ExecuteNonQuery();
UpdateChannelStatistics(conn);
UpdateChannelStatistics();
}
public List<Channel> GetAllChannels() {
var list = new List<Channel>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT id, server, name, parent_id, position, topic, nsfw FROM channels");
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
list.Add(new Channel {
Id = reader.GetUint64(0),
Server = reader.GetUint64(1),
Id = (ulong) reader.GetInt64(0),
Server = (ulong) reader.GetInt64(1),
Name = reader.GetString(2),
ParentId = reader.IsDBNull(3) ? null : reader.GetUint64(3),
ParentId = reader.IsDBNull(3) ? null : (ulong) reader.GetInt64(3),
Position = reader.IsDBNull(4) ? null : reader.GetInt32(4),
Topic = reader.IsDBNull(5) ? null : reader.GetString(5),
Nsfw = reader.IsDBNull(6) ? null : reader.GetBoolean(6)
@@ -164,185 +111,179 @@ namespace DHT.Server.Database.Sqlite {
}
public void AddUsers(User[] users) {
using var conn = pool.Take();
using var tx = conn.BeginTransaction();
using var cmd = conn.Upsert("users", new[] {
("id", SqliteType.Integer),
("name", SqliteType.Text),
("avatar_url", SqliteType.Text),
("discriminator", SqliteType.Text)
"id", "name", "avatar_url", "discriminator"
});
var userParams = cmd.Parameters;
userParams.Add(":id", SqliteType.Integer);
userParams.Add(":name", SqliteType.Text);
userParams.Add(":avatar_url", SqliteType.Text);
userParams.Add(":discriminator", SqliteType.Text);
foreach (var user in users) {
cmd.Set(":id", user.Id);
cmd.Set(":name", user.Name);
cmd.Set(":avatar_url", user.AvatarUrl);
cmd.Set(":discriminator", user.Discriminator);
userParams.Set(":id", user.Id);
userParams.Set(":name", user.Name);
userParams.Set(":avatar_url", user.AvatarUrl);
userParams.Set(":discriminator", user.Discriminator);
cmd.ExecuteNonQuery();
}
tx.Commit();
UpdateUserStatistics(conn);
UpdateUserStatistics();
}
public List<User> GetAllUsers() {
var perf = log.Start();
var list = new List<User>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT id, name, avatar_url, discriminator FROM users");
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
list.Add(new User {
Id = reader.GetUint64(0),
Id = (ulong) reader.GetInt64(0),
Name = reader.GetString(1),
AvatarUrl = reader.IsDBNull(2) ? null : reader.GetString(2),
Discriminator = reader.IsDBNull(3) ? null : reader.GetString(3)
});
}
perf.End();
return list;
}
public void AddMessages(Message[] messages) {
static SqliteCommand DeleteByMessageId(ISqliteConnection conn, string tableName) {
return conn.Delete(tableName, ("message_id", SqliteType.Integer));
}
using var tx = conn.BeginTransaction();
using var messageCmd = conn.Upsert("messages", new[] {
"message_id", "sender_id", "channel_id", "text", "timestamp", "edit_timestamp", "replied_to_id"
});
static void ExecuteDeleteByMessageId(SqliteCommand cmd, object id) {
cmd.Set(":message_id", id);
cmd.ExecuteNonQuery();
}
bool addedAttachments = false;
using var messageRawCmd = conn.Upsert("messages_raw", new[] {
"message_id", "json"
});
using (var conn = pool.Take()) {
using var tx = conn.BeginTransaction();
using var deleteAttachmentsCmd = conn.Command("DELETE FROM attachments WHERE message_id = :message_id");
using var attachmentCmd = conn.Insert("attachments", new[] {
"message_id", "attachment_id", "name", "type", "url", "size"
});
using var messageCmd = conn.Upsert("messages", new[] {
("message_id", SqliteType.Integer),
("sender_id", SqliteType.Integer),
("channel_id", SqliteType.Integer),
("text", SqliteType.Text),
("timestamp", SqliteType.Integer)
});
using var deleteEmbedsCmd = conn.Command("DELETE FROM embeds WHERE message_id = :message_id");
using var embedCmd = conn.Insert("embeds", new[] {
"message_id", "json"
});
using var deleteEditTimestampCmd = DeleteByMessageId(conn, "edit_timestamps");
using var deleteRepliedToCmd = DeleteByMessageId(conn, "replied_to");
using var deleteReactionsCmd = conn.Command("DELETE FROM reactions WHERE message_id = :message_id");
using var reactionCmd = conn.Insert("reactions", new[] {
"message_id", "emoji_id", "emoji_name", "emoji_flags", "count"
});
using var deleteAttachmentsCmd = DeleteByMessageId(conn, "attachments");
using var deleteEmbedsCmd = DeleteByMessageId(conn, "embeds");
using var deleteReactionsCmd = DeleteByMessageId(conn, "reactions");
var messageParams = messageCmd.Parameters;
messageParams.Add(":message_id", SqliteType.Integer);
messageParams.Add(":sender_id", SqliteType.Integer);
messageParams.Add(":channel_id", SqliteType.Integer);
messageParams.Add(":text", SqliteType.Text);
messageParams.Add(":timestamp", SqliteType.Integer);
messageParams.Add(":edit_timestamp", SqliteType.Integer);
messageParams.Add(":replied_to_id", SqliteType.Integer);
using var editTimestampCmd = conn.Insert("edit_timestamps", new [] {
("message_id", SqliteType.Integer),
("edit_timestamp", SqliteType.Integer)
});
var messageRawParams = messageRawCmd.Parameters;
messageRawParams.Add(":message_id", SqliteType.Integer);
messageRawParams.Add(":json", SqliteType.Blob);
using var repliedToCmd = conn.Insert("replied_to", new [] {
("message_id", SqliteType.Integer),
("replied_to_id", SqliteType.Integer)
});
var deleteAttachmentsParams = deleteAttachmentsCmd.Parameters;
deleteAttachmentsParams.Add(":message_id", SqliteType.Integer);
using var attachmentCmd = conn.Insert("attachments", new[] {
("message_id", SqliteType.Integer),
("attachment_id", SqliteType.Integer),
("name", SqliteType.Text),
("type", SqliteType.Text),
("url", SqliteType.Text),
("size", SqliteType.Integer)
});
var attachmentParams = attachmentCmd.Parameters;
attachmentParams.Add(":message_id", SqliteType.Integer);
attachmentParams.Add(":attachment_id", SqliteType.Integer);
attachmentParams.Add(":name", SqliteType.Text);
attachmentParams.Add(":type", SqliteType.Text);
attachmentParams.Add(":url", SqliteType.Text);
attachmentParams.Add(":size", SqliteType.Integer);
using var embedCmd = conn.Insert("embeds", new[] {
("message_id", SqliteType.Integer),
("json", SqliteType.Text)
});
var deleteEmbedsParams = deleteEmbedsCmd.Parameters;
deleteEmbedsParams.Add(":message_id", SqliteType.Integer);
using var reactionCmd = conn.Insert("reactions", new[] {
("message_id", SqliteType.Integer),
("emoji_id", SqliteType.Integer),
("emoji_name", SqliteType.Text),
("emoji_flags", SqliteType.Integer),
("count", SqliteType.Integer)
});
var embedParams = embedCmd.Parameters;
embedParams.Add(":message_id", SqliteType.Integer);
embedParams.Add(":json", SqliteType.Text);
foreach (var message in messages) {
object messageId = message.Id;
var deleteReactionsParams = deleteReactionsCmd.Parameters;
deleteReactionsParams.Add(":message_id", SqliteType.Integer);
messageCmd.Set(":message_id", messageId);
messageCmd.Set(":sender_id", message.Sender);
messageCmd.Set(":channel_id", message.Channel);
messageCmd.Set(":text", message.Text);
messageCmd.Set(":timestamp", message.Timestamp);
messageCmd.ExecuteNonQuery();
var reactionParams = reactionCmd.Parameters;
reactionParams.Add(":message_id", SqliteType.Integer);
reactionParams.Add(":emoji_id", SqliteType.Integer);
reactionParams.Add(":emoji_name", SqliteType.Text);
reactionParams.Add(":emoji_flags", SqliteType.Integer);
reactionParams.Add(":count", SqliteType.Integer);
ExecuteDeleteByMessageId(deleteEditTimestampCmd, messageId);
ExecuteDeleteByMessageId(deleteRepliedToCmd, messageId);
var brotli = new Brotli(4096);
ExecuteDeleteByMessageId(deleteAttachmentsCmd, messageId);
ExecuteDeleteByMessageId(deleteEmbedsCmd, messageId);
ExecuteDeleteByMessageId(deleteReactionsCmd, messageId);
foreach (var message in messages) {
object messageId = message.Id;
if (message.EditTimestamp is {} timestamp) {
editTimestampCmd.Set(":message_id", messageId);
editTimestampCmd.Set(":edit_timestamp", timestamp);
editTimestampCmd.ExecuteNonQuery();
}
messageParams.Set(":message_id", messageId);
messageParams.Set(":sender_id", message.Sender);
messageParams.Set(":channel_id", message.Channel);
messageParams.Set(":text", message.Text);
messageParams.Set(":timestamp", message.Timestamp);
messageParams.Set(":edit_timestamp", message.EditTimestamp);
messageParams.Set(":replied_to_id", message.RepliedToId);
messageCmd.ExecuteNonQuery();
if (message.RepliedToId is {} repliedToId) {
repliedToCmd.Set(":message_id", messageId);
repliedToCmd.Set(":replied_to_id", repliedToId);
repliedToCmd.ExecuteNonQuery();
}
if (message.RawJson is {} json) {
messageRawParams.Set(":message_id", messageId);
messageRawParams.Set(":json", brotli.Compress(Encoding.UTF8.GetBytes(json)));
messageRawCmd.ExecuteNonQuery();
}
if (!message.Attachments.IsEmpty) {
addedAttachments = true;
foreach (var attachment in message.Attachments) {
attachmentCmd.Set(":message_id", messageId);
attachmentCmd.Set(":attachment_id", attachment.Id);
attachmentCmd.Set(":name", attachment.Name);
attachmentCmd.Set(":type", attachment.Type);
attachmentCmd.Set(":url", attachment.Url);
attachmentCmd.Set(":size", attachment.Size);
attachmentCmd.ExecuteNonQuery();
}
}
deleteAttachmentsParams.Set(":message_id", messageId);
deleteAttachmentsCmd.ExecuteNonQuery();
if (!message.Embeds.IsEmpty) {
foreach (var embed in message.Embeds) {
embedCmd.Set(":message_id", messageId);
embedCmd.Set(":json", embed.Json);
embedCmd.ExecuteNonQuery();
}
}
deleteEmbedsParams.Set(":message_id", messageId);
deleteEmbedsCmd.ExecuteNonQuery();
if (!message.Reactions.IsEmpty) {
foreach (var reaction in message.Reactions) {
reactionCmd.Set(":message_id", messageId);
reactionCmd.Set(":emoji_id", reaction.EmojiId);
reactionCmd.Set(":emoji_name", reaction.EmojiName);
reactionCmd.Set(":emoji_flags", (int) reaction.EmojiFlags);
reactionCmd.Set(":count", reaction.Count);
reactionCmd.ExecuteNonQuery();
}
deleteReactionsParams.Set(":message_id", messageId);
deleteReactionsCmd.ExecuteNonQuery();
if (!message.Attachments.IsEmpty) {
foreach (var attachment in message.Attachments) {
attachmentParams.Set(":message_id", messageId);
attachmentParams.Set(":attachment_id", attachment.Id);
attachmentParams.Set(":name", attachment.Name);
attachmentParams.Set(":type", attachment.Type);
attachmentParams.Set(":url", attachment.Url);
attachmentParams.Set(":size", attachment.Size);
attachmentCmd.ExecuteNonQuery();
}
}
tx.Commit();
if (!message.Embeds.IsEmpty) {
foreach (var embed in message.Embeds) {
embedParams.Set(":message_id", messageId);
embedParams.Set(":json", embed.Json);
embedCmd.ExecuteNonQuery();
}
}
if (!message.Reactions.IsEmpty) {
foreach (var reaction in message.Reactions) {
reactionParams.Set(":message_id", messageId);
reactionParams.Set(":emoji_id", reaction.EmojiId);
reactionParams.Set(":emoji_name", reaction.EmojiName);
reactionParams.Set(":emoji_flags", (int) reaction.EmojiFlags);
reactionParams.Set(":count", reaction.Count);
reactionCmd.ExecuteNonQuery();
}
}
}
totalMessagesComputer.Recompute();
if (addedAttachments) {
totalAttachmentsComputer.Recompute();
}
tx.Commit();
UpdateMessageStatistics();
}
public int CountMessages(MessageFilter? filter = null) {
using var conn = pool.Take();
using var cmd = conn.Command("SELECT COUNT(*) FROM messages" + filter.GenerateWhereClause());
using var reader = cmd.ExecuteReader();
@@ -350,217 +291,68 @@ namespace DHT.Server.Database.Sqlite {
}
public List<Message> GetMessages(MessageFilter? filter = null) {
var perf = log.Start();
var list = new List<Message>();
var attachments = GetAllAttachments();
var embeds = GetAllEmbeds();
var reactions = GetAllReactions();
using var conn = pool.Take();
using var cmd = conn.Command(@"
SELECT m.message_id, m.sender_id, m.channel_id, m.text, m.timestamp, et.edit_timestamp, rt.replied_to_id
FROM messages m
LEFT JOIN edit_timestamps et ON m.message_id = et.message_id
LEFT JOIN replied_to rt ON m.message_id = rt.message_id" + filter.GenerateWhereClause("m"));
var list = new List<Message>();
using var cmd = conn.Command("SELECT message_id, sender_id, channel_id, text, timestamp, edit_timestamp, replied_to_id FROM messages" + filter.GenerateWhereClause());
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
ulong id = reader.GetUint64(0);
ulong id = (ulong) reader.GetInt64(0);
list.Add(new Message {
Id = id,
Sender = reader.GetUint64(1),
Channel = reader.GetUint64(2),
Sender = (ulong) reader.GetInt64(1),
Channel = (ulong) reader.GetInt64(2),
Text = reader.GetString(3),
Timestamp = reader.GetInt64(4),
EditTimestamp = reader.IsDBNull(5) ? null : reader.GetInt64(5),
RepliedToId = reader.IsDBNull(6) ? null : reader.GetUint64(6),
RepliedToId = reader.IsDBNull(6) ? null : (ulong) reader.GetInt64(6),
Attachments = attachments.GetListOrNull(id)?.ToImmutableArray() ?? ImmutableArray<Attachment>.Empty,
Embeds = embeds.GetListOrNull(id)?.ToImmutableArray() ?? ImmutableArray<Embed>.Empty,
Reactions = reactions.GetListOrNull(id)?.ToImmutableArray() ?? ImmutableArray<Reaction>.Empty
});
}
perf.End();
return list;
}
public HashSet<ulong> GetMessageIds(MessageFilter? filter = null) {
var perf = log.Start();
var ids = new HashSet<ulong>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT message_id FROM messages" + filter.GenerateWhereClause());
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
ids.Add(reader.GetUint64(0));
public void RemoveMessages(MessageFilter filter, MessageFilterRemovalMode mode) {
var whereClause = filter.GenerateWhereClause(invert: mode == MessageFilterRemovalMode.KeepMatching);
if (string.IsNullOrEmpty(whereClause)) {
return;
}
perf.End();
return ids;
}
// Rider is being stupid...
StringBuilder build = new StringBuilder()
.Append("DELETE ")
.Append("FROM messages")
.Append(whereClause);
public void RemoveMessages(MessageFilter filter, FilterRemovalMode mode) {
var perf = log.Start();
DeleteFromTable("messages", filter.GenerateWhereClause(invert: mode == FilterRemovalMode.KeepMatching));
totalMessagesComputer.Recompute();
perf.End();
}
public int CountAttachments(AttachmentFilter? filter = null) {
using var conn = pool.Take();
using var cmd = conn.Command("SELECT COUNT(DISTINCT url) FROM attachments a" + filter.GenerateWhereClause("a"));
using var reader = cmd.ExecuteReader();
return reader.Read() ? reader.GetInt32(0) : 0;
}
public void AddDownload(Data.Download download) {
using var conn = pool.Take();
using var cmd = conn.Upsert("downloads", new[] {
("url", SqliteType.Text),
("status", SqliteType.Integer),
("size", SqliteType.Integer),
("blob", SqliteType.Blob)
});
cmd.Set(":url", download.Url);
cmd.Set(":status", (int) download.Status);
cmd.Set(":size", download.Size);
cmd.Set(":blob", download.Data);
using var cmd = conn.Command(build.ToString());
cmd.ExecuteNonQuery();
totalDownloadsComputer.Recompute();
}
public List<Data.Download> GetDownloadsWithoutData() {
var list = new List<Data.Download>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT url, status, size FROM downloads");
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
string url = reader.GetString(0);
var status = (DownloadStatus) reader.GetInt32(1);
ulong size = reader.GetUint64(2);
list.Add(new Data.Download(url, status, size));
}
return list;
}
public Data.Download GetDownloadWithData(Data.Download download) {
using var conn = pool.Take();
using var cmd = conn.Command("SELECT blob FROM downloads WHERE url = :url");
cmd.AddAndSet(":url", SqliteType.Text, download.Url);
using var reader = cmd.ExecuteReader();
if (reader.Read() && !reader.IsDBNull(0)) {
return download.WithData((byte[]) reader["blob"]);
}
else {
return download;
}
}
public void EnqueueDownloadItems(AttachmentFilter? filter = null) {
using var conn = pool.Take();
using var cmd = conn.Command("INSERT INTO downloads (url, status, size) SELECT a.url, :enqueued, MAX(a.size) FROM attachments a" + filter.GenerateWhereClause("a") + " GROUP BY a.url");
cmd.AddAndSet(":enqueued", SqliteType.Integer, (int) DownloadStatus.Enqueued);
cmd.ExecuteNonQuery();
}
public List<DownloadItem> GetEnqueuedDownloadItems(int count) {
var list = new List<DownloadItem>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT url, size FROM downloads WHERE status = :enqueued LIMIT :limit");
cmd.AddAndSet(":enqueued", SqliteType.Integer, (int) DownloadStatus.Enqueued);
cmd.AddAndSet(":limit", SqliteType.Integer, Math.Max(0, count));
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
list.Add(new DownloadItem {
Url = reader.GetString(0),
Size = reader.GetUint64(1)
});
}
return list;
}
public void RemoveDownloadItems(DownloadItemFilter? filter, FilterRemovalMode mode) {
DeleteFromTable("downloads", filter.GenerateWhereClause(invert: mode == FilterRemovalMode.KeepMatching));
totalDownloadsComputer.Recompute();
}
public DownloadStatusStatistics GetDownloadStatusStatistics() {
static void LoadUndownloadedStatistics(ISqliteConnection conn, DownloadStatusStatistics result) {
using var cmd = conn.Command("SELECT IFNULL(COUNT(size), 0), IFNULL(SUM(size), 0) FROM (SELECT MAX(a.size) size FROM attachments a WHERE a.url NOT IN (SELECT d.url FROM downloads d) GROUP BY a.url)");
using var reader = cmd.ExecuteReader();
if (reader.Read()) {
result.SkippedCount = reader.GetInt32(0);
result.SkippedSize = reader.GetUint64(1);
}
}
static void LoadSuccessStatistics(ISqliteConnection conn, DownloadStatusStatistics result) {
using var cmd = conn.Command(@"SELECT
IFNULL(SUM(CASE WHEN status = :enqueued THEN 1 ELSE 0 END), 0),
IFNULL(SUM(CASE WHEN status = :enqueued THEN size ELSE 0 END), 0),
IFNULL(SUM(CASE WHEN status = :success THEN 1 ELSE 0 END), 0),
IFNULL(SUM(CASE WHEN status = :success THEN size ELSE 0 END), 0),
IFNULL(SUM(CASE WHEN status != :enqueued AND status != :success THEN 1 ELSE 0 END), 0),
IFNULL(SUM(CASE WHEN status != :enqueued AND status != :success THEN size ELSE 0 END), 0)
FROM downloads");
cmd.AddAndSet(":enqueued", SqliteType.Integer, (int) DownloadStatus.Enqueued);
cmd.AddAndSet(":success", SqliteType.Integer, (int) DownloadStatus.Success);
using var reader = cmd.ExecuteReader();
if (reader.Read()) {
result.EnqueuedCount = reader.GetInt32(0);
result.EnqueuedSize = reader.GetUint64(1);
result.SuccessfulCount = reader.GetInt32(2);
result.SuccessfulSize = reader.GetUint64(3);
result.FailedCount = reader.GetInt32(4);
result.FailedSize = reader.GetUint64(5);
}
}
var result = new DownloadStatusStatistics();
using var conn = pool.Take();
LoadUndownloadedStatistics(conn, result);
LoadSuccessStatistics(conn, result);
return result;
UpdateMessageStatistics();
}
private MultiDictionary<ulong, Attachment> GetAllAttachments() {
var dict = new MultiDictionary<ulong, Attachment>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT message_id, attachment_id, name, type, url, size FROM attachments");
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
ulong messageId = reader.GetUint64(0);
ulong messageId = (ulong) reader.GetInt64(0);
dict.Add(messageId, new Attachment {
Id = reader.GetUint64(1),
Id = (ulong) reader.GetInt64(1),
Name = reader.GetString(2),
Type = reader.IsDBNull(3) ? null : reader.GetString(3),
Url = reader.GetString(4),
Size = reader.GetUint64(5)
Size = (ulong) reader.GetInt64(5)
});
}
@@ -570,12 +362,11 @@ FROM downloads");
private MultiDictionary<ulong, Embed> GetAllEmbeds() {
var dict = new MultiDictionary<ulong, Embed>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT message_id, json FROM embeds");
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
ulong messageId = reader.GetUint64(0);
ulong messageId = (ulong) reader.GetInt64(0);
dict.Add(messageId, new Embed {
Json = reader.GetString(1)
@@ -588,15 +379,14 @@ FROM downloads");
private MultiDictionary<ulong, Reaction> GetAllReactions() {
var dict = new MultiDictionary<ulong, Reaction>();
using var conn = pool.Take();
using var cmd = conn.Command("SELECT message_id, emoji_id, emoji_name, emoji_flags, count FROM reactions");
using var reader = cmd.ExecuteReader();
while (reader.Read()) {
ulong messageId = reader.GetUint64(0);
ulong messageId = (ulong) reader.GetInt64(0);
dict.Add(messageId, new Reaction {
EmojiId = reader.IsDBNull(1) ? null : reader.GetUint64(1),
EmojiId = reader.IsDBNull(1) ? null : (ulong) reader.GetInt64(1),
EmojiName = reader.IsDBNull(2) ? null : reader.GetString(2),
EmojiFlags = (EmojiFlags) reader.GetInt16(3),
Count = reader.GetInt32(4)
@@ -606,62 +396,24 @@ FROM downloads");
return dict;
}
private void DeleteFromTable(string table, string whereClause) {
// Rider is being stupid...
StringBuilder build = new StringBuilder()
.Append("DELETE ")
.Append("FROM ")
.Append(table)
.Append(whereClause);
using var conn = pool.Take();
using var cmd = conn.Command(build.ToString());
cmd.ExecuteNonQuery();
private void UpdateServerStatistics() {
using var cmd = conn.Command("SELECT COUNT(*) FROM servers");
Statistics.TotalServers = cmd.ExecuteScalar() as long? ?? 0;
}
public void Vacuum() {
using var conn = pool.Take();
using var cmd = conn.Command("VACUUM");
cmd.ExecuteNonQuery();
private void UpdateChannelStatistics() {
using var cmd = conn.Command("SELECT COUNT(*) FROM channels");
Statistics.TotalChannels = cmd.ExecuteScalar() as long? ?? 0;
}
private void UpdateServerStatistics(ISqliteConnection conn) {
Statistics.TotalServers = conn.SelectScalar("SELECT COUNT(*) FROM servers") as long? ?? 0;
private void UpdateUserStatistics() {
using var cmd = conn.Command("SELECT COUNT(*) FROM users");
Statistics.TotalUsers = cmd.ExecuteScalar() as long? ?? 0;
}
private void UpdateChannelStatistics(ISqliteConnection conn) {
Statistics.TotalChannels = conn.SelectScalar("SELECT COUNT(*) FROM channels") as long? ?? 0;
}
private void UpdateUserStatistics(ISqliteConnection conn) {
Statistics.TotalUsers = conn.SelectScalar("SELECT COUNT(*) FROM users") as long? ?? 0;
}
private long ComputeMessageStatistics() {
using var conn = pool.Take();
return conn.SelectScalar("SELECT COUNT(*) FROM messages") as long? ?? 0L;
}
private void UpdateMessageStatistics(long totalMessages) {
Statistics.TotalMessages = totalMessages;
}
private long ComputeAttachmentStatistics() {
using var conn = pool.Take();
return conn.SelectScalar("SELECT COUNT(DISTINCT url) FROM attachments") as long? ?? 0L;
}
private void UpdateAttachmentStatistics(long totalAttachments) {
Statistics.TotalAttachments = totalAttachments;
}
private long ComputeDownloadStatistics() {
using var conn = pool.Take();
return conn.SelectScalar("SELECT COUNT(*) FROM downloads") as long? ?? 0L;
}
private void UpdateDownloadStatistics(long totalDownloads) {
Statistics.TotalDownloads = totalDownloads;
private void UpdateMessageStatistics() {
using var cmd = conn.Command("SELECT COUNT(*) FROM messages");
Statistics.TotalMessages = cmd.ExecuteScalar() as long? ?? 0L;
}
}
}

83
app/Server/Database/Sqlite/SqliteFilters.cs
View File

@@ -1,83 +0,0 @@
using System;
using System.Collections.Generic;
using System.Linq;
using DHT.Server.Data;
using DHT.Server.Data.Filters;
using DHT.Server.Database.Sqlite.Utils;
namespace DHT.Server.Database.Sqlite {
static class SqliteFilters {
public static string GenerateWhereClause(this MessageFilter? filter, string? tableAlias = null, bool invert = false) {
if (filter == null) {
return "";
}
var where = new SqliteWhereGenerator(tableAlias, invert);
if (filter.StartDate != null) {
where.AddCondition("timestamp >= " + new DateTimeOffset(filter.StartDate.Value).ToUnixTimeMilliseconds());
}
if (filter.EndDate != null) {
where.AddCondition("timestamp <= " + new DateTimeOffset(filter.EndDate.Value).ToUnixTimeMilliseconds());
}
if (filter.ChannelIds != null) {
where.AddCondition("channel_id IN (" + string.Join(",", filter.ChannelIds) + ")");
}
if (filter.UserIds != null) {
where.AddCondition("sender_id IN (" + string.Join(",", filter.UserIds) + ")");
}
if (filter.MessageIds != null) {
where.AddCondition("message_id IN (" + string.Join(",", filter.MessageIds) + ")");
}
return where.Generate();
}
public static string GenerateWhereClause(this AttachmentFilter? filter, string? tableAlias = null, bool invert = false) {
if (filter == null) {
return "";
}
var where = new SqliteWhereGenerator(tableAlias, invert);
if (filter.MaxBytes != null) {
where.AddCondition("size <= " + filter.MaxBytes);
}
if (filter.DownloadItemRule == AttachmentFilter.DownloadItemRules.OnlyNotPresent) {
where.AddCondition("url NOT IN (SELECT url FROM downloads)");
}
else if (filter.DownloadItemRule == AttachmentFilter.DownloadItemRules.OnlyPresent) {
where.AddCondition("url IN (SELECT url FROM downloads)");
}
return where.Generate();
}
public static string GenerateWhereClause(this DownloadItemFilter? filter, string? tableAlias = null, bool invert = false) {
if (filter == null) {
return "";
}
var where = new SqliteWhereGenerator(tableAlias, invert);
if (filter.IncludeStatuses != null) {
where.AddCondition("status IN (" + filter.IncludeStatuses.In() + ")");
}
if (filter.ExcludeStatuses != null) {
where.AddCondition("status NOT IN (" + filter.ExcludeStatuses.In() + ")");
}
return where.Generate();
}
private static string In(this ISet<DownloadStatus> statuses) {
return string.Join(",", statuses.Select(static status => (int) status));
}
}
}

46
app/Server/Database/Sqlite/SqliteMessageFilter.cs Normal file
View File

@@ -0,0 +1,46 @@
using System;
using System.Collections.Generic;
using DHT.Server.Data.Filters;
namespace DHT.Server.Database.Sqlite {
static class SqliteMessageFilter {
public static string GenerateWhereClause(this MessageFilter? filter, bool invert = false) {
if (filter == null) {
return "";
}
List<string> conditions = new();
if (filter.StartDate != null) {
conditions.Add("timestamp >= " + new DateTimeOffset(filter.StartDate.Value).ToUnixTimeMilliseconds());
}
if (filter.EndDate != null) {
conditions.Add("timestamp <= " + new DateTimeOffset(filter.EndDate.Value).ToUnixTimeMilliseconds());
}
if (filter.ChannelIds != null) {
conditions.Add("channel_id IN (" + string.Join(",", filter.ChannelIds) + ")");
}
if (filter.UserIds != null) {
conditions.Add("sender_id IN (" + string.Join(",", filter.UserIds) + ")");
}
if (filter.MessageIds != null) {
conditions.Add("message_id IN (" + string.Join(",", filter.MessageIds) + ")");
}
if (conditions.Count == 0) {
return "";
}
if (invert) {
return " WHERE NOT (" + string.Join(" AND ", conditions) + ")";
}
else {
return " WHERE " + string.Join(" AND ", conditions);
}
}
}
}

40
app/Server/Database/Sqlite/SqliteUtils.cs Normal file
View File

@@ -0,0 +1,40 @@
using System;
using System.Linq;
using Microsoft.Data.Sqlite;
namespace DHT.Server.Database.Sqlite {
static class SqliteUtils {
public static SqliteCommand Command(this SqliteConnection conn, string sql) {
var cmd = conn.CreateCommand();
cmd.CommandText = sql;
return cmd;
}
public static SqliteCommand Insert(this SqliteConnection conn, string tableName, string[] columns) {
string columnNames = string.Join(',', columns);
string columnParams = string.Join(',', columns.Select(static c => ':' + c));
return conn.Command("INSERT INTO " + tableName + " (" + columnNames + ")" +
"VALUES (" + columnParams + ")");
}
public static SqliteCommand Upsert(this SqliteConnection conn, string tableName, string[] columns) {
string columnNames = string.Join(',', columns);
string columnParams = string.Join(',', columns.Select(static c => ':' + c));
string columnUpdates = string.Join(',', columns.Skip(1).Select(static c => c + " = excluded." + c));
return conn.Command("INSERT INTO " + tableName + " (" + columnNames + ")" +
"VALUES (" + columnParams + ")" +
"ON CONFLICT (" + columns[0] + ")" +
"DO UPDATE SET " + columnUpdates);
}
public static void AddAndSet(this SqliteParameterCollection parameters, string key, object? value) {
parameters.AddWithValue(key, value ?? DBNull.Value);
}
public static void Set(this SqliteParameterCollection parameters, string key, object? value) {
parameters[key].Value = value ?? DBNull.Value;
}
}
}

8
app/Server/Database/Sqlite/Utils/ISqliteConnection.cs
View File

@@ -1,8 +0,0 @@
using System;
using Microsoft.Data.Sqlite;
namespace DHT.Server.Database.Sqlite.Utils {
interface ISqliteConnection : IDisposable {
SqliteConnection InnerConnection { get; }
}
}

117
app/Server/Database/Sqlite/Utils/SqliteConnectionPool.cs
View File

@@ -1,117 +0,0 @@
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Threading;
using DHT.Utils.Logging;
using Microsoft.Data.Sqlite;
namespace DHT.Server.Database.Sqlite.Utils {
sealed class SqliteConnectionPool : IDisposable {
private static string GetConnectionString(SqliteConnectionStringBuilder connectionStringBuilder) {
connectionStringBuilder.Pooling = false;
return connectionStringBuilder.ToString();
}
private readonly object monitor = new ();
private readonly Random rand = new ();
private volatile bool isDisposed;
private readonly BlockingCollection<PooledConnection> free = new (new ConcurrentStack<PooledConnection>());
private readonly List<PooledConnection> used;
public SqliteConnectionPool(SqliteConnectionStringBuilder connectionStringBuilder, int poolSize) {
var connectionString = GetConnectionString(connectionStringBuilder);
for (int i = 0; i < poolSize; i++) {
var conn = new SqliteConnection(connectionString);
conn.Open();
var pooledConn = new PooledConnection(this, conn);
using (var cmd = pooledConn.Command("PRAGMA journal_mode=WAL")) {
cmd.ExecuteNonQuery();
}
free.Add(pooledConn);
}
used = new List<PooledConnection>(poolSize);
}
private void ThrowIfDisposed() {
if (isDisposed) {
throw new ObjectDisposedException(nameof(SqliteConnectionPool));
}
}
public ISqliteConnection Take() {
PooledConnection? conn = null;
while (conn == null) {
ThrowIfDisposed();
lock (monitor) {
if (free.TryTake(out conn, TimeSpan.FromMilliseconds(rand.Next(100, 200)))) {
used.Add(conn);
break;
}
else {
Log.ForType<SqliteConnectionPool>().Warn("Thread " + Thread.CurrentThread.ManagedThreadId + " is starving for connections.");
}
}
}
return conn;
}
private void Return(PooledConnection conn) {
ThrowIfDisposed();
lock (monitor) {
if (used.Remove(conn)) {
free.Add(conn);
}
}
}
public void Dispose() {
if (isDisposed) {
return;
}
isDisposed = true;
lock (monitor) {
while (free.TryTake(out var conn)) {
Close(conn.InnerConnection);
}
foreach (var conn in used) {
Close(conn.InnerConnection);
}
free.Dispose();
used.Clear();
}
}
private static void Close(SqliteConnection conn) {
conn.Close();
conn.Dispose();
}
private sealed class PooledConnection : ISqliteConnection {
public SqliteConnection InnerConnection { get; }
private readonly SqliteConnectionPool pool;
public PooledConnection(SqliteConnectionPool pool, SqliteConnection conn) {
this.pool = pool;
this.InnerConnection = conn;
}
void IDisposable.Dispose() {
pool.Return(this);
}
}
}
}

71
app/Server/Database/Sqlite/Utils/SqliteExtensions.cs
View File

@@ -1,71 +0,0 @@
using System;
using System.Linq;
using Microsoft.Data.Sqlite;
namespace DHT.Server.Database.Sqlite.Utils {
static class SqliteExtensions {
public static SqliteCommand Command(this ISqliteConnection conn, string sql) {
var cmd = conn.InnerConnection.CreateCommand();
cmd.CommandText = sql;
return cmd;
}
public static SqliteTransaction BeginTransaction(this ISqliteConnection conn) {
return conn.InnerConnection.BeginTransaction();
}
public static object? SelectScalar(this ISqliteConnection conn, string sql) {
using var cmd = conn.Command(sql);
return cmd.ExecuteScalar();
}
public static SqliteCommand Insert(this ISqliteConnection conn, string tableName, (string Name, SqliteType Type)[] columns) {
string columnNames = string.Join(',', columns.Select(static c => c.Name));
string columnParams = string.Join(',', columns.Select(static c => ':' + c.Name));
var cmd = conn.Command("INSERT INTO " + tableName + " (" + columnNames + ")" +
"VALUES (" + columnParams + ")");
CreateParameters(cmd, columns);
return cmd;
}
public static SqliteCommand Upsert(this ISqliteConnection conn, string tableName, (string Name, SqliteType Type)[] columns) {
string columnNames = string.Join(',', columns.Select(static c => c.Name));
string columnParams = string.Join(',', columns.Select(static c => ':' + c.Name));
string columnUpdates = string.Join(',', columns.Skip(1).Select(static c => c.Name + " = excluded." + c.Name));
var cmd = conn.Command("INSERT INTO " + tableName + " (" + columnNames + ")" +
"VALUES (" + columnParams + ")" +
"ON CONFLICT (" + columns[0].Name + ")" +
"DO UPDATE SET " + columnUpdates);
CreateParameters(cmd, columns);
return cmd;
}
public static SqliteCommand Delete(this ISqliteConnection conn, string tableName, (string Name, SqliteType Type) column) {
var cmd = conn.Command("DELETE FROM " + tableName + " WHERE " + column.Name + " = :" + column.Name);
CreateParameters(cmd, new [] { column });
return cmd;
}
private static void CreateParameters(SqliteCommand cmd, (string Name, SqliteType Type)[] columns) {
foreach (var (name, type) in columns) {
cmd.Parameters.Add(":" + name, type);
}
}
public static void AddAndSet(this SqliteCommand cmd, string key, SqliteType type, object? value) {
cmd.Parameters.Add(key, type).Value = value ?? DBNull.Value;
}
public static void Set(this SqliteCommand cmd, string key, object? value) {
cmd.Parameters[key].Value = value ?? DBNull.Value;
}
public static ulong GetUint64(this SqliteDataReader reader, int ordinal) {
return (ulong) reader.GetInt64(ordinal);
}
}
}

31
app/Server/Database/Sqlite/Utils/SqliteWhereGenerator.cs
View File

@@ -1,31 +0,0 @@
using System.Collections.Generic;
namespace DHT.Server.Database.Sqlite.Utils {
sealed class SqliteWhereGenerator {
private readonly string? tableAlias;
private readonly bool invert;
private readonly List<string> conditions = new ();
public SqliteWhereGenerator(string? tableAlias, bool invert) {
this.tableAlias = tableAlias;
this.invert = invert;
}
public void AddCondition(string condition) {
conditions.Add(tableAlias == null ? condition : tableAlias + '.' + condition);
}
public string Generate() {
if (conditions.Count == 0) {
return "";
}
if (invert) {
return " WHERE NOT (" + string.Join(" AND ", conditions) + ")";
}
else {
return " WHERE " + string.Join(" AND ", conditions);
}
}
}
}

129
app/Server/Download/BackgroundDownloadThread.cs
View File

@@ -1,129 +0,0 @@
using System;
using System.Collections.Concurrent;
using System.Net;
using System.Threading;
using DHT.Server.Database;
using DHT.Utils.Logging;
using DHT.Utils.Models;
namespace DHT.Server.Download {
public sealed class BackgroundDownloadThread : BaseModel {
private static readonly Log Log = Log.ForType<BackgroundDownloadThread>();
public event EventHandler<DownloadItem>? OnItemFinished {
add => parameters.OnItemFinished += value;
remove => parameters.OnItemFinished -= value;
}
public event EventHandler? OnServerStopped {
add => parameters.OnServerStopped += value;
remove => parameters.OnServerStopped -= value;
}
private readonly CancellationTokenSource cancellationTokenSource;
private readonly ThreadInstance.Parameters parameters;
public BackgroundDownloadThread(IDatabaseFile db) {
this.cancellationTokenSource = new CancellationTokenSource();
this.parameters = new ThreadInstance.Parameters(db, cancellationTokenSource);
var thread = new Thread(new ThreadInstance().Work) {
Name = "DHT download thread"
};
thread.Start(parameters);
}
public void StopThread() {
try {
cancellationTokenSource.Cancel();
} catch (ObjectDisposedException) {
Log.Warn("Attempted to stop background download thread after the cancellation token has been disposed.");
}
}
private sealed class ThreadInstance {
private const int QueueSize = 32;
public sealed class Parameters {
public event EventHandler<DownloadItem>? OnItemFinished;
public event EventHandler? OnServerStopped;
public IDatabaseFile Db { get; }
public CancellationTokenSource CancellationTokenSource { get; }
public Parameters(IDatabaseFile db, CancellationTokenSource cancellationTokenSource) {
Db = db;
CancellationTokenSource = cancellationTokenSource;
}
public void FireOnItemFinished(DownloadItem item) {
OnItemFinished?.Invoke(null, item);
}
public void FireOnServerStopped() {
OnServerStopped?.Invoke(null, EventArgs.Empty);
}
}
private readonly WebClient client = new ();
public ThreadInstance() {
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36";
}
public void Work(object? obj) {
var parameters = (Parameters) obj!;
var cancellationTokenSource = parameters.CancellationTokenSource;
var cancellationToken = cancellationTokenSource.Token;
var db = parameters.Db;
var queue = new ConcurrentQueue<DownloadItem>();
cancellationToken.Register(client.CancelAsync);
try {
while (!cancellationToken.IsCancellationRequested) {
FillQueue(db, queue, cancellationToken);
while (!cancellationToken.IsCancellationRequested && queue.TryDequeue(out var item)) {
var url = item.Url;
Log.Debug("Downloading " + url + "...");
try {
db.AddDownload(Data.Download.NewSuccess(url, client.DownloadData(url)));
} catch (WebException e) {
db.AddDownload(Data.Download.NewFailure(url, e.Response is HttpWebResponse response ? response.StatusCode : null, item.Size));
Log.Error(e);
} finally {
parameters.FireOnItemFinished(item);
}
}
}
} catch (OperationCanceledException) {
//
} catch (ObjectDisposedException) {
//
} finally {
cancellationTokenSource.Dispose();
parameters.FireOnServerStopped();
}
}
private static void FillQueue(IDatabaseFile db, ConcurrentQueue<DownloadItem> queue, CancellationToken cancellationToken) {
while (!cancellationToken.IsCancellationRequested && queue.IsEmpty) {
var newItems = db.GetEnqueuedDownloadItems(QueueSize);
if (newItems.Count == 0) {
Thread.Sleep(TimeSpan.FromMilliseconds(50));
}
else {
foreach (var item in newItems) {
queue.Enqueue(item);
}
}
}
}
}
}
}

6
app/Server/Download/DownloadItem.cs
View File

@@ -1,6 +0,0 @@
namespace DHT.Server.Download {
public readonly struct DownloadItem {
public string Url { get; init; }
public ulong Size { get; init; }
}
}

9
app/Server/Endpoints/TrackMessagesEndpoint.cs
View File

@@ -1,6 +1,5 @@
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Net;
using System.Text.Json;
@@ -9,7 +8,6 @@ using DHT.Server.Data;
using DHT.Server.Data.Filters;
using DHT.Server.Database;
using DHT.Server.Service;
using DHT.Utils.Collections;
using DHT.Utils.Http;
using Microsoft.AspNetCore.Http;
@@ -52,19 +50,16 @@ namespace DHT.Server.Endpoints {
RepliedToId = json.HasKey("repliedToId") ? json.RequireSnowflake("repliedToId", path) : null,
Attachments = json.HasKey("attachments") ? ReadAttachments(json.RequireArray("attachments", path + ".attachments"), path + ".attachments[]").ToImmutableArray() : ImmutableArray<Attachment>.Empty,
Embeds = json.HasKey("embeds") ? ReadEmbeds(json.RequireArray("embeds", path + ".embeds"), path + ".embeds[]").ToImmutableArray() : ImmutableArray<Embed>.Empty,
Reactions = json.HasKey("reactions") ? ReadReactions(json.RequireArray("reactions", path + ".reactions"), path + ".reactions[]").ToImmutableArray() : ImmutableArray<Reaction>.Empty
Reactions = json.HasKey("reactions") ? ReadReactions(json.RequireArray("reactions", path + ".reactions"), path + ".reactions[]").ToImmutableArray() : ImmutableArray<Reaction>.Empty,
RawJson = json.HasKey("raw") ? json.RequireString("raw", path) : null
};
[SuppressMessage("ReSharper", "ConvertToLambdaExpression")]
private static IEnumerable<Attachment> ReadAttachments(JsonElement.ArrayEnumerator array, string path) => array.Select(ele => new Attachment {
Id = ele.RequireSnowflake("id", path),
Name = ele.RequireString("name", path),
Type = ele.HasKey("type") ? ele.RequireString("type", path) : null,
Url = ele.RequireString("url", path),
Size = (ulong) ele.RequireLong("size", path)
}).DistinctByKeyStable(static attachment => {
// Some Discord messages have duplicate attachments with the same id for unknown reasons.
return attachment.Id;
});
private static IEnumerable<Embed> ReadEmbeds(JsonElement.ArrayEnumerator array, string path) => array.Select(ele => new Embed {

Some files were not shown because too many files have changed in this diff Show More