34 Commits
Author SHA1 Message Date
OneWay cab2dc742e fix: restore boundary progress publishes and animate the bar at constant velocity 2026-09-06 18:14:53 +08:00
OneWay 962605e2b4 fix: latch terminal notification state and post pipeline progress on one thread
A late ImportState Success snapshot collected after the success
notification overwrote it with an ongoing progress record. Post
progress on Main behind a volatile terminal latch; notifications go
through NotificationPort, the pipeline lives in the coordinator, and
the service finish is injected.
2026-09-06 17:05:46 +08:00
OneWay d360ecfb2b fix: defer import errors until the copy loop finishes
Errors and skips are collected while the copy loop runs and reported
once it ends. The final import state is exhaustive - success, already
added, invalid name, or nothing supported found - so every import run
reaches a terminal state and the pipeline notification always resolves
instead of lingering as an indeterminate progress bar.
2026-09-06 17:05:46 +08:00
OneWay 78754d2c75 fix: tie back arrow to pipeline idle state
Computed from operation states in SettingsViewModel instead of a fixed
delay; cancel and error release the arrow immediately.
2026-09-05 05:55:56 +08:00
OneWay 3567f50ad3 perf(about): parse aboutlibraries.json off the main thread
Bump AboutLibraries plugin to 15.0.4 -> 15.2.0; parse
aboutlibraries.json on Dispatchers.IO via produceState (header renders
immediately, rows fill in); drop animateContentSize from rows.
2026-09-05 00:37:41 +08:00
OneWay c79be491cc fix: open Settings immediately on cold start when no dictionaries
Seed and synchronously read hadNoDictionaries before setContent so
Settings shows from the first frame until the startup scan confirms.
2026-09-04 19:09:19 +08:00
OneWay 0f3d4b965c fix: key search result rows by dictionary, offset and word
Paging result rows had no keys and were matched by position: appended
pages and query diffs could not reuse row state. Keys derived from
dictionaryPath+offset+word are stable per row, and the paging source
collapses duplicate entries for one dictionary card: case and diacritic
alias variants of a headword share the normalized word, so entries are
deduplicated by dictionaryPath+word+offset before emitting.
2026-09-05 03:00:44 +08:00
OneWay e6f9f7f900 fix: hide dictionary rows from accessibility while mutations blocked
invisibleToUser() removes blocked content from the accessibility tree;
mergeDescendants only regrouped traversal order.
2026-09-04 18:04:32 +08:00
OneWay e5650f7247 fix: re-check mutation-block inside gesture coroutines
Stale drag-tick coroutines launched before the flip no-op at execution
time instead of moving the row after the reset.
2026-09-04 13:28:27 +08:00
OneWay ccbd037507 refactor: unify swipe reset to snap before clearing offset
Snap the Animatable first, clear rawOffset after - no frame with the
row still translated while the delete affordance is hidden.
2026-09-04 02:05:59 +08:00
OneWay 1907e3004d fix: key dictionary list items by path
Swipe state follows its row instead of migrating to a neighbor on
delete or reorder.
2026-09-04 01:53:06 +08:00
OneWay 84800cf2b0 refactor: isolate dictionary rows from mutation-block recomposition
Delete-block flag becomes a () -> Boolean provider read at state/gesture
time, so flipping it no longer recomposes the whole Dictionaries block.
2026-09-04 14:59:59 +08:00
OneWay 33cc386015 refactor: replace deprecated currentWindowAdaptiveInfo with V2 2026-09-02 16:18:39 +08:00
OneWay 07031c7ac1 refactor: expose state flows via asStateFlow
Drops the explicit getters and an unused import, and lets an emptyList
type argument be inferred.
2026-09-02 12:55:48 +08:00
OneWay 5a7ba69b28 chore: remove unused index comparison and decode helpers
IndexSearcher.compareEntryGroups / IndexComparisonSummary /
IndexComparisonException / readEntryOrNull and AboutLibrariesData.decode
have no call sites left in the app.
2026-09-02 11:30:51 +08:00
OneWay bfe4721c6b fix: reset dictionary progress bar between indexing operations
monotonicTargetProgress is held in remember and only ever grows via
maxOf, so when a new indexing starts after a previous one reached a
higher value the bar is left at the old level while the percent label
starts at 0. Snap it to the current target when the section becomes
visible so the bar follows the new operation from its first frame.
2026-09-02 10:56:32 +08:00
OneWay 097bb23ce9 fix: cancel stale progress notification on process start
Notifications belong to the package and outlive the process: after the
system kills the foreground service mid-download the frozen progress
notification stays in the shade forever. A fresh process means no
service is running, so any leftover notification is stale.
2026-09-02 08:37:43 +08:00
OneWay 5126c19f3c perf: mark Compose state types as @Immutable
Compose compiler reports showed DownloadState's sealed root as Uncertain
and DownloadProgressState/SettingsUiState as runtime stability, so
skippability checks were deferred to runtime. The annotations make them
compile-time stable and turn any future mutable field into an explicit
contract break.
2026-09-01 17:28:17 +08:00
OneWay 70faf71f99 chore: drop the unused indexing_label string 2026-09-01 13:36:23 +08:00
OneWay 9fcd7914dc refactor: slim IndexingProgress to a producer-supplied [0,1] progress
perFileProgress was never populated and its progress branch was
unreachable; currentFile, currentIndex, currentFileProgress and label
were write-only (no consumer read them); progressPercent had no
callers; the -1f aggregate sentinel is gone along with the fallback
branches that only fired when the tracker was absent.
2026-09-01 13:36:23 +08:00
OneWay bd7f597986 fix: report Idle instead of fake Success when nothing is downloaded
The early return skipped the terminal-state delay and reset, leaving
a phantom 100% Success (and a stuck "indexing" bar plus a success
notification) for downloads that never happened.
2026-09-05 10:03:54 +08:00
OneWay 98ef99ab53 fix: probe source availability before showing download progress
Instant server failures (404/HTML error page) no longer flash a 0%
progress bar; the check mirrors the downloader's previous-day fallback
and tolerates transient network errors, which still go through the
downloader's retry path.
2026-09-05 10:03:54 +08:00
OneWay 43823e3bfa Fix: clear search results instantly on query clear
Skipped the 300ms debounce for blank queries so the results list clears immediately; typed queries keep the debounce.
2026-08-30 03:36:27 +08:00
OneWay bab03b6c73 fix: retry dictionary downloads on network-switch errors
Downloads killed by WiFi/mobile transitions failed with the generic
"Dictionary download failed" message because UnknownHostException,
ConnectException, unexpected end of stream, and closed-socket errors
were not classified as network errors and were never retried.
Classify them by exception type and message so the retry loop
engages and exhaustion reports the network-specific message.

Bump OkHttp 5.4.0 to 5.5.0: a throwing system proxy selector now
falls back to a direct connection instead of failing the call,
complementing the download resilience fix.
2026-08-30 00:39:04 +08:00
OneWay d4e6302274 docs: update README and Fastlane descriptions 2026-08-25 23:59:58 +08:00
OneWay bc3fa85b9d Release 1.5.0
- The dictionary update card now returns after a failed download
- Fixed headwords and articles with escaped characters being displayed incorrectly
- Fixed slow rendering of articles containing many bracket characters
2026-08-18 00:16:27 +03:00
OneWay 457bbe43b6 Unescape any escaped character in article text
Apply the general DSL rule \X -> X instead of unescaping only brackets
and parentheses, so sequences like \! no longer show the backslash.
2026-08-18 00:15:53 +03:00
OneWay f751ac0d36 Harden DSL parsing against escapes and malformed tags
Fixed escape-aware headword parsing for \{, \}, \[ and trailing backslash.
Fixed O(n^2) tag scan and nested bracket handling in article rendering.
2026-08-16 11:13:46 +03:00
OneWay 910bc8313c Release 1.4.0
- Fixed matching source URLs to installed dictionary files, so deleted dictionaries can be downloaded again from the same URL
2026-08-06 14:26:03 +03:00
OneWay 9d18d0c927 Release 1.3.0
- Blocked swipe-to-delete on dictionary rows while a download, extract,
  import or indexing pass is running. Deleting mid-update removed the
  dictionary's source from preferences while the pipeline kept its own
  snapshot, so the post-update rescan re-created the dictionary with its
  update source permanently lost.
- Fixed the dictionary icon leaving the row while swiping to delete: the
  item now clips to its bounds and the delete button is a static overlay
  instead of a counter-translated child of the sliding row.
2026-08-02 11:09:47 +03:00
OneWay 69bbb97b2a Release 1.2.1
- Fixed the dictionary update card taking too long to reappear after
  cancelling a download (DownloadManager).
- Bump agp 9.3.0 -> 9.3.1; migrate the release build type from
  isMinifyEnabled/isShrinkResources/proguardFiles to
  optimization { enable = true } with keep rules moved to
  app/src/release/keepRules/proguard-rules.keep.
2026-07-31 10:47:29 +03:00
OneWay c8adf537d4 Add app icon to fastlane metadata 2026-07-30 19:46:12 +03:00
OneWay 84612b26f2 Release 1.2.0
- Fixed a crash on broad queries (e.g. a single letter across several
  dictionaries): the results list keyed items by dictionary
  path/offset/word, and duplicate keys are illegal in a Compose list;
  drop the explicit key, let Paging identify items by position instead
  (SearchResultsList).
- Fixed a disabled dictionary reactivating itself on the next app
  launch: the active flag is now persisted through
  PreferencesManager/DataStore and reapplied whenever the dictionary
  list is rebuilt (LocalDictionaryRepository).
- Search results no longer stop at the first ~100-200 matches: the
  ranked head page hands back a cursor per unfinished prefix range,
  and IndexEntryPagingSource pages the rest straight from the index,
  merged alphabetically across dictionaries, until every match is
  surfaced (IndexSearcher, IndexEntryPagingSource).
- Added diacritic-insensitive search: folded index entries are merged
  into the primary index as aliases, so accented and plain spellings
  find each other (DslIndexer, IndexSearcher).
- Fixed indexing failures on dictionaries with dense non-Latin text: a
  truncated multi-byte sequence at a sample boundary was misread as
  invalid UTF-8 and the whole file fell back to UTF-16LE
  (DslCharsetDetector).
2026-07-22 16:29:16 +03:00
OneWay a5f3af83a4 Fix dictionary update flow and rebuild the search field on Material 3
Fixed the update card reappearing for a moment after tapping it: the tap
re-ran the status check and republished NeedsUpdate before the download
service reported Loading, so the card flashed back in that gap.

Fixed the update card never returning after cancelling a download: both the
Cancelled branch and cancelDownload() forced the status to UpToDate, and the
card's tap is the only thing that re-evaluates it. The status is now
re-derived on the Cancelled to Idle transition, once the partially downloaded
archive - which carries today's date and would otherwise be read as a current
dictionary - has been cleaned up.

Fixed the download progress bar staying at 0% far too long: progress was
published in 3-percentage-point buckets and then scaled by the download stage
weight, so the bar only moved after about 7% of the archive had transferred.

Rebuilt the search field on the Material 3 TextField instead of a hand-rolled
BasicTextField inside a Surface, so it picks up the standard container, cursor
and icon treatment. Dictionary entry titles in the search field, the results
list and the article bar now share a dictionaryTitleLarge style that keeps the
font padding, so their line height lines up with the Material 3 text field.
2026-07-13 19:20:25 +08:00
72 changed files with 1876 additions and 980 deletions
+6 -6
View File
@@ -29,7 +29,7 @@ The app stores dictionaries as DictZip files and builds compact binary indexes f
- Random-access article loading from DictZip dictionaries. - Random-access article loading from DictZip dictionaries.
- Background processing with progress shown in Settings and notifications. - Background processing with progress shown in Settings and notifications.
- Dictionary management with enable, disable, update, and delete actions. - Dictionary management with enable, disable, update, and delete actions.
- Material 3 interface with light, dark, and system themes. - Material 3 interface with dynamic color, light, dark, and system themes.
- English and Russian localization. - English and Russian localization.
- About screen with open source license information. - About screen with open source license information.
@@ -94,16 +94,16 @@ The main search screen remains usable as long as at least one indexed dictionary
## Tech Stack ## Tech Stack
- Kotlin 2.4.0 - Kotlin 2.4.10
- Jetpack Compose (BOM 2026.06.01) - Jetpack Compose (BOM 2026.08.00)
- Material 3 - Material 3
- Coroutines and Flow 1.11.0 - Coroutines and Flow 1.11.0
- DataStore Preferences 1.2.1 - DataStore Preferences 1.2.1
- Paging 3.5.0 - Paging 3.5.1
- OkHttp 5.4.0 - OkHttp 5.4.0
- kotlinx.serialization 1.11.0 - kotlinx.serialization 1.11.0
- AboutLibraries 15.0.3 metadata generation - AboutLibraries 15.0.4 metadata generation
- Android Gradle Plugin 9.2.1 - Android Gradle Plugin 9.3.1
## Build ## Build
+5 -14
View File
@@ -92,14 +92,8 @@ android {
applicationId = "com.example.research" applicationId = "com.example.research"
minSdk = project.property("minSdk").toString().toInt() minSdk = project.property("minSdk").toString().toInt()
targetSdk = project.property("targetSdk").toString().toInt() targetSdk = project.property("targetSdk").toString().toInt()
versionCode = 2 versionCode = 8
versionName = "1.1.0" versionName = "1.5.0"
val isFdroid = project.hasProperty("fdroid")
if (isFdroid) {
applicationIdSuffix = ".fdroid"
versionNameSuffix = "-fdroid"
}
vectorDrawables { vectorDrawables {
useSupportLibrary = true useSupportLibrary = true
@@ -120,12 +114,9 @@ android {
abiFilters += "arm64-v8a" abiFilters += "arm64-v8a"
} }
isMinifyEnabled = true optimization {
isShrinkResources = true enable = true
proguardFiles( }
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro"
)
} }
} }
@@ -0,0 +1,45 @@
package com.example.research.core.performance
import android.os.Trace
import java.util.concurrent.atomic.AtomicInteger
object ReSearchTrace {
const val SEARCH_PAGING = "ReSearch/Search/Paging"
const val SEARCH_DIRECT = "ReSearch/Search/Direct"
const val ARTICLE_PARSE = "ReSearch/Article/Parse"
const val ARTICLE_READ = "ReSearch/Article/Read"
const val ARTICLE_PRE_MEASURE = "ReSearch/Article/PreMeasure"
const val DICTIONARY_DOWNLOAD = "ReSearch/Dictionary/Download"
const val DICTIONARY_EXTRACT = "ReSearch/Dictionary/Extract"
const val DICTIONARY_INDEX = "ReSearch/Dictionary/Index"
const val DICTIONARY_IMPORT = "ReSearch/Dictionary/Import"
const val FLOW_UPDATE_CARD_APPEAR = "ReSearch/Flow/UpdateCardAppear"
const val FLOW_UPDATING = "ReSearch/Flow/Updating"
const val FLOW_PROGRESS_DISAPPEAR = "ReSearch/Flow/ProgressDisappear"
private val nextCookie = AtomicInteger()
@PublishedApi
internal val isAndroidRuntime = System.getProperty("java.vm.name") == "Dalvik"
inline fun <T> section(name: String, block: () -> T): T {
if (!isAndroidRuntime) return block()
Trace.beginSection(name)
return try {
block()
} finally {
Trace.endSection()
}
}
suspend fun <T> asyncSection(name: String, block: suspend () -> T): T {
if (!isAndroidRuntime) return block()
val cookie = nextCookie.incrementAndGet()
Trace.beginAsyncSection(name, cookie)
return try {
block()
} finally {
Trace.endAsyncSection(name, cookie)
}
}
}
@@ -1,5 +1,8 @@
package com.example.research package com.example.research
import androidx.compose.runtime.Immutable
@Immutable
sealed interface DictionaryStatus { sealed interface DictionaryStatus {
data object Unknown : DictionaryStatus data object Unknown : DictionaryStatus
data object Checking : DictionaryStatus data object Checking : DictionaryStatus
@@ -83,6 +83,7 @@ class MainActivity : ComponentActivity() {
val app = application as ReSearchApplication val app = application as ReSearchApplication
val initialTheme = loadInitialTheme(app) val initialTheme = loadInitialTheme(app)
val initialLanguage = loadInitialLanguage(app) val initialLanguage = loadInitialLanguage(app)
val initialHadNoDictionaries = loadInitialHadNoDictionaries(app)
handleIntent(intent) handleIntent(intent)
setContent { setContent {
@@ -104,7 +105,8 @@ class MainActivity : ComponentActivity() {
) { ) {
AppNavigation( AppNavigation(
searchViewModel = searchViewModel, searchViewModel = searchViewModel,
settingsViewModel = settingsViewModel settingsViewModel = settingsViewModel,
seedNoDictionaries = initialHadNoDictionaries
) )
} }
} }
@@ -165,6 +167,16 @@ class MainActivity : ComponentActivity() {
} }
} }
private fun loadInitialHadNoDictionaries(app: ReSearchApplication): Boolean {
return try {
runBlocking {
app.preferencesManager.hadNoDictionaries.first()
}
} catch (e: Exception) {
true
}
}
override fun onNewIntent(intent: Intent) { override fun onNewIntent(intent: Intent) {
super.onNewIntent(intent) super.onNewIntent(intent)
handleIntent(intent) handleIntent(intent)
@@ -2,6 +2,7 @@ package com.example.research
import android.app.Application import android.app.Application
import com.example.research.common.progress.DictionaryProgressStateHolder import com.example.research.common.progress.DictionaryProgressStateHolder
import com.example.research.common.util.NotificationHelper
import com.example.research.data.local.preferences.PreferencesManager import com.example.research.data.local.preferences.PreferencesManager
import com.example.research.data.repository.LocalDictionaryRepository import com.example.research.data.repository.LocalDictionaryRepository
import com.example.research.feature.download.DownloadManager import com.example.research.feature.download.DownloadManager
@@ -36,16 +37,23 @@ class ReSearchApplication : Application() {
private set private set
lateinit var dictionaryProgressStateHolder: DictionaryProgressStateHolder lateinit var dictionaryProgressStateHolder: DictionaryProgressStateHolder
private set private set
lateinit var dictionaryProgressPresenter: com.example.research.common.progress.DictionaryProgressPresenter
private set
lateinit var dictionaryPipelineCoordinator: com.example.research.common.progress.DictionaryPipelineCoordinator
private set
private val applicationScope = CoroutineScope(SupervisorJob() + Dispatchers.Default) private val applicationScope = CoroutineScope(SupervisorJob() + Dispatchers.Default)
override fun onCreate() { override fun onCreate() {
super.onCreate() super.onCreate()
val notificationHelper = NotificationHelper(this)
notificationHelper.cancelProgressNotification()
preferencesManager = PreferencesManager(this) preferencesManager = PreferencesManager(this)
applicationScope.launch(Dispatchers.IO) { applicationScope.launch(Dispatchers.IO) {
preferencesManager.sanitizeDictionarySources() preferencesManager.sanitizeDictionarySources()
} }
localDictionaryRepository = LocalDictionaryRepository(this) localDictionaryRepository = LocalDictionaryRepository(this, preferencesManager = preferencesManager)
okHttpClient = OkHttpClient.Builder() okHttpClient = OkHttpClient.Builder()
.protocols(listOf(Protocol.HTTP_1_1)) .protocols(listOf(Protocol.HTTP_1_1))
@@ -62,7 +70,8 @@ class ReSearchApplication : Application() {
downloadManager = DownloadManager( downloadManager = DownloadManager(
dictionaryRepository = downloadDictionaryRepository, dictionaryRepository = downloadDictionaryRepository,
unknownErrorMessage = getString(R.string.unknown_error) unknownErrorMessage = getString(R.string.unknown_error),
sourceUnavailableMessage = getString(R.string.download_file_not_found)
) )
dictionaryImportManager = DictionaryImportManager( dictionaryImportManager = DictionaryImportManager(
@@ -76,6 +85,40 @@ class ReSearchApplication : Application() {
localDictionaryRepository = localDictionaryRepository, localDictionaryRepository = localDictionaryRepository,
) )
dictionaryProgressPresenter = com.example.research.common.progress.DictionaryProgressPresenter(
scope = applicationScope,
progressSnapshot = dictionaryProgressStateHolder.progressSnapshot,
notifications = notificationHelper,
)
dictionaryPipelineCoordinator = com.example.research.common.progress.DictionaryPipelineCoordinator(
scope = applicationScope,
steps = com.example.research.common.progress.DefaultReindexingSteps(
localDictionaryRepository = localDictionaryRepository,
downloadDictionaryRepository = downloadDictionaryRepository,
),
importOperations = dictionaryImportManager,
ui = dictionaryProgressPresenter,
dictionaryPath = { preferencesManager.dictionaryPath },
finishService = {
val intent = android.content.Intent(
this,
com.example.research.feature.download.service.DictionaryForegroundService::class.java
).apply {
action = com.example.research.feature.download.service.DictionaryForegroundService.ACTION_FINISH
}
try {
startService(intent)
} catch (_: Exception) {
}
},
)
dictionaryImportManager.onFlowStarted = { dictionaryProgressPresenter.beginFlow() }
dictionaryImportManager.onTerminal = dictionaryPipelineCoordinator::onImportTerminal
downloadManager.onFlowStarted = { dictionaryProgressPresenter.beginFlow() }
downloadManager.onDownloadSuccess = dictionaryPipelineCoordinator::onDownloadSuccess
downloadManager.onTerminal = dictionaryPipelineCoordinator::onDownloadTerminal
} }
override fun onTrimMemory(level: Int) { override fun onTrimMemory(level: Int) {
@@ -0,0 +1,131 @@
package com.example.research.common.progress
import com.example.research.core.util.OperationResult
import com.example.research.feature.download.model.DownloadState
import com.example.research.ui.settings.ImportState
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
import java.io.File
import kotlin.time.Duration.Companion.seconds
class DictionaryPipelineCoordinator(
private val scope: CoroutineScope,
private val steps: ReindexingSteps,
private val importOperations: ImportFlowOperations,
private val ui: PipelineTerminalUi,
private val dictionaryPath: () -> String,
private val finishService: () -> Unit,
private val launchDispatcher: CoroutineDispatcher = Dispatchers.Main,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO,
) {
private var pipelineJob: Job? = null
fun onImportTerminal(state: ImportState) {
when (state) {
is ImportState.Success -> runPipeline(isImport = true)
is ImportState.Error -> {
ui.showError()
finishService()
}
else -> {}
}
}
fun onDownloadSuccess() = runPipeline(isImport = false)
fun onDownloadTerminal(state: DownloadState) {
when (state) {
is DownloadState.Error -> {
ui.showError()
finishService()
}
is DownloadState.Cancelled -> {
ui.cancelProgress()
finishService()
}
else -> {}
}
}
fun cancelPipeline() {
pipelineJob?.cancel()
pipelineJob = null
}
private fun runPipeline(isImport: Boolean) {
pipelineJob?.cancel()
pipelineJob = scope.launch(launchDispatcher) {
val dir = if (isImport) File(dictionaryPath()) else null
val filesBeforeReindex = dir?.listFiles()?.map { it.name }?.toSet() ?: emptySet()
try {
triggerReindexing(isImport)
if (isImport) {
importOperations.getAndClearImportedFiles()
importOperations.clearImportState()
ui.showImportSuccess()
} else {
ui.showDownloadSuccess()
}
} catch (e: CancellationException) {
if (isImport) {
val filesToCleanup = importOperations.getAndClearImportedFiles()
filesToCleanup.forEach { file ->
try {
if (file.exists()) file.delete()
} catch (_: Exception) {
}
}
cleanupNewFiles(dir, filesBeforeReindex)
}
throw e
} catch (e: Exception) {
ui.showError()
} finally {
finishService()
}
}
}
private suspend fun triggerReindexing(isImport: Boolean) = withContext(ioDispatcher) {
steps.waitForIndexingCompletion()
delay(1.seconds)
val path = dictionaryPath()
if (isImport) {
importOperations.updateExtractionProgress(0f)
}
steps.extractArchives(onProgress = { progress ->
if (isImport) {
importOperations.updateExtractionProgress(progress)
}
})
val result = steps.scanDirectory(path)
if (result is OperationResult.Success && result.data > 0) {
steps.warmupIndexes()
steps.performAllCleanup()
}
if (isImport) {
importOperations.markImportPipelineSuccess()
}
}
private fun cleanupNewFiles(dir: File?, filesBeforeSnapshot: Set<String>) {
try {
dir?.listFiles()?.forEach { file ->
if (file.name !in filesBeforeSnapshot) {
try {
file.delete()
} catch (_: Exception) {
}
}
}
} catch (_: Exception) {
}
}
}
@@ -0,0 +1,56 @@
package com.example.research.common.progress
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.launch
class DictionaryProgressPresenter(
private val scope: CoroutineScope,
private val progressSnapshot: StateFlow<DictionaryProgressModel.Snapshot?>,
private val notifications: NotificationPort,
mainDispatcher: CoroutineDispatcher = Dispatchers.Main,
) : PipelineTerminalUi {
@Volatile
private var terminalActive = false
init {
scope.launch(mainDispatcher) {
progressSnapshot.collect { snapshot ->
when {
snapshot == null -> {
if (!terminalActive && notifications.hasActiveProgressNotification()) {
notifications.cancelProgressNotification()
}
}
!terminalActive -> notifications.showProgress(snapshot.percent, snapshot.titleRes)
}
}
}
}
fun beginFlow() {
terminalActive = false
}
override fun showImportSuccess() {
notifications.showImportSuccess()
terminalActive = true
}
override fun showDownloadSuccess() {
notifications.showDownloadSuccess()
terminalActive = true
}
override fun showError() {
notifications.showError()
terminalActive = true
}
override fun cancelProgress() {
terminalActive = false
notifications.cancelProgressNotification()
}
}
@@ -0,0 +1,10 @@
package com.example.research.common.progress
interface NotificationPort {
fun showProgress(percent: Int, titleRes: Int)
fun showImportSuccess()
fun showDownloadSuccess()
fun showError()
fun cancelProgressNotification()
fun hasActiveProgressNotification(): Boolean
}
@@ -0,0 +1,65 @@
package com.example.research.common.progress
import com.example.research.core.util.OperationResult
import com.example.research.data.repository.LocalDictionaryRepository
import com.example.research.feature.download.repository.DictionaryRepository
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.first
import java.io.File
import kotlin.time.Duration.Companion.milliseconds
interface PipelineTerminalUi {
fun showImportSuccess()
fun showDownloadSuccess()
fun showError()
fun cancelProgress()
}
interface ImportFlowOperations {
fun updateExtractionProgress(progress: Float)
fun markImportPipelineSuccess()
fun clearImportState()
fun getAndClearImportedFiles(): List<File>
}
interface ReindexingSteps {
suspend fun waitForIndexingCompletion(): Boolean
suspend fun extractArchives(onProgress: (Float) -> Unit)
suspend fun scanDirectory(path: String): OperationResult<Int>
suspend fun warmupIndexes()
suspend fun performAllCleanup()
}
class DefaultReindexingSteps(
private val localDictionaryRepository: LocalDictionaryRepository,
private val downloadDictionaryRepository: DictionaryRepository,
) : ReindexingSteps {
override suspend fun waitForIndexingCompletion(): Boolean {
if (!localDictionaryRepository.indexingProgress.first().isIndexing) return true
var waitCount = 0
while (localDictionaryRepository.indexingProgress.first().isIndexing && waitCount < MAX_POLL) {
delay(POLL_INTERVAL)
waitCount++
}
return waitCount < MAX_POLL
}
override suspend fun extractArchives(onProgress: (Float) -> Unit) {
downloadDictionaryRepository.extractArchives(onProgress = onProgress)
}
override suspend fun scanDirectory(path: String): OperationResult<Int> =
localDictionaryRepository.scanDirectory(path)
override suspend fun warmupIndexes() = localDictionaryRepository.warmupIndexes()
override suspend fun performAllCleanup() {
downloadDictionaryRepository.performAllCleanup()
}
private companion object {
const val MAX_POLL = 100
val POLL_INTERVAL = 200.milliseconds
}
}
@@ -1,6 +1,7 @@
package com.example.research.common.ui.theme package com.example.research.common.ui.theme
import androidx.compose.material3.Typography import androidx.compose.material3.Typography
import androidx.compose.ui.text.PlatformTextStyle
import androidx.compose.ui.text.TextStyle import androidx.compose.ui.text.TextStyle
import androidx.compose.ui.text.font.FontFamily import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight import androidx.compose.ui.text.font.FontWeight
@@ -85,3 +86,8 @@ val Typography = Typography(
letterSpacing = 0.5.sp letterSpacing = 0.5.sp
) )
) )
val Typography.dictionaryTitleLarge: TextStyle
get() = titleLarge.copy(
platformStyle = PlatformTextStyle(includeFontPadding = true)
)
@@ -13,7 +13,7 @@ object DateUtils {
return dateFormat.format(calendar.time) return dateFormat.format(calendar.time)
} }
fun extractDateFromFileName(fileName: String): String? { fun extractDateFromFileName(fileName: String): String? {
val regex = Regex("""_(\d{6})\.(gz|dsl|idx|dsl\.dz|dsl\.idx|dsl\.gz)$""") val regex = Regex("""_(\d{6})\.(gz|dsl|idx|idx\.fold|dsl\.dz|dsl\.idx|dsl\.idx\.fold|dsl\.gz)$""")
return regex.find(fileName)?.groupValues?.get(1) return regex.find(fileName)?.groupValues?.get(1)
} }
fun isDateBefore(date1: String, date2: String): Boolean { fun isDateBefore(date1: String, date2: String): Boolean {
@@ -7,8 +7,9 @@ import androidx.core.app.*
import androidx.core.content.ContextCompat import androidx.core.content.ContextCompat
import com.example.research.MainActivity import com.example.research.MainActivity
import com.example.research.R import com.example.research.R
import com.example.research.common.progress.NotificationPort
import com.example.research.feature.download.receiver.DownloadCancelReceiver import com.example.research.feature.download.receiver.DownloadCancelReceiver
class NotificationHelper(private val context: Context) { class NotificationHelper(private val context: Context) : NotificationPort {
companion object { companion object {
const val CHANNEL_ID = "download_progress_channel" const val CHANNEL_ID = "download_progress_channel"
const val NOTIFICATION_ID = 1 const val NOTIFICATION_ID = 1
@@ -64,6 +65,22 @@ class NotificationHelper(private val context: Context) {
android.Manifest.permission.POST_NOTIFICATIONS android.Manifest.permission.POST_NOTIFICATIONS
) == android.content.pm.PackageManager.PERMISSION_GRANTED ) == android.content.pm.PackageManager.PERMISSION_GRANTED
} }
override fun showProgress(percent: Int, titleRes: Int) {
showUnifiedProgressNotification(
title = context.getString(titleRes),
contentText = "$percent%",
progressPercent = percent,
)
}
override fun showDownloadSuccess() = showSuccessNotification()
override fun showImportSuccess() = showImportSuccessNotification()
override fun showError() = showErrorNotification()
override fun cancelProgressNotification() = cancelNotification()
fun showSuccessNotification() { fun showSuccessNotification() {
if (!canShowNotification()) return if (!canShowNotification()) return
lastNotificationKey = 0 lastNotificationKey = 0
@@ -93,6 +110,8 @@ class NotificationHelper(private val context: Context) {
notificationManager.cancel(NOTIFICATION_ID) notificationManager.cancel(NOTIFICATION_ID)
} }
override fun hasActiveProgressNotification(): Boolean = lastNotificationKey != 0
fun showUnifiedProgressNotification( fun showUnifiedProgressNotification(
title: String, title: String,
contentText: String, contentText: String,
@@ -13,7 +13,8 @@ data class IndexEntry(
val offset: ArticleOffset = ArticleOffset.ZERO, val offset: ArticleOffset = ArticleOffset.ZERO,
val length: ArticleLength = ArticleLength.ZERO, val length: ArticleLength = ArticleLength.ZERO,
val dictionaryName: String = "", val dictionaryName: String = "",
val dictionaryPath: String = "" val dictionaryPath: String = "",
val isAlias: Boolean = false
) : Comparable<IndexEntry> { ) : Comparable<IndexEntry> {
init { init {
require(word.isNotBlank()) { "Word cannot be blank" } require(word.isNotBlank()) { "Word cannot be blank" }
@@ -1,42 +1,16 @@
package com.example.research.core.domain.model package com.example.research.core.domain.model
import kotlin.math.roundToInt
import androidx.compose.runtime.Immutable
/** /**
* Snapshot of an ongoing indexing operation. * Snapshot of an ongoing indexing operation.
* *
* [progress] is the authoritative [0f, 1f] completion value and is O(1) to * [progress] is the authoritative [0f, 1f] completion value: the repository
* compute: the repository's internal `ProgressTracker` maintains a running * aggregates a file-size-weighted sum across files incrementally while
* sum across files incrementally and stores it in [aggregateSum], avoiding * indexing, so reading it is O(1).
* an O(n) walk over [perFileProgress] on every read.
* [perFileProgress] is kept for diagnostics; callers should prefer
* [progress] / [progressPercent].
*/ */
@Immutable
data class IndexingProgress( data class IndexingProgress(
val currentFile: String = "",
val currentIndex: Int = 0,
val totalFiles: Int = 0,
val isIndexing: Boolean = false, val isIndexing: Boolean = false,
val currentFileProgress: Float = 0f, val progress: Float = 0f,
val label: String = "", )
val perFileProgress: Map<String, Float> = emptyMap(),
/** Pre-aggregated sum in [0f, totalFiles] supplied by the producer; -1f = unknown. */
val aggregateSum: Float = -1f,
) {
val progress: Float
get() = if (totalFiles > 0) {
when {
aggregateSum >= 0f -> (aggregateSum / totalFiles).coerceIn(0f, 1f)
perFileProgress.isNotEmpty() -> {
val totalProgress = perFileProgress.values.sum()
(totalProgress / totalFiles).coerceIn(0f, 1f)
}
else -> {
val completedFiles = (currentIndex - 1).coerceAtLeast(0)
((completedFiles + currentFileProgress) / totalFiles).coerceIn(0f, 1f)
}
}
} else 0f
val progressPercent: Int
get() = (progress * 100).roundToInt().coerceIn(0, 100)
}
@@ -0,0 +1,27 @@
package com.example.research.core.domain.usecase
import com.example.research.core.domain.model.Dictionary
import com.example.research.core.domain.model.DictionarySource
import java.io.File
object DictionarySourceFileMatcher {
fun matches(source: DictionarySource, fileName: String): Boolean =
DictionarySource.matchesDictionaryFile(source.urlTemplate, fileName)
fun matches(source: DictionarySource, dictionary: Dictionary): Boolean =
matches(source, File(dictionary.path).name)
fun installedSources(
sources: List<DictionarySource>,
dictionaries: List<Dictionary>,
): List<DictionarySource> = sources.filter { source ->
dictionaries.any { dictionary -> matches(source, dictionary) }
}
fun installedSourcesForFileNames(
sources: List<DictionarySource>,
fileNames: Collection<String>,
): List<DictionarySource> = sources.filter { source ->
fileNames.any { fileName -> matches(source, fileName) }
}
}
@@ -43,12 +43,4 @@ class DictionarySourceValidator {
return ValidationResult.Valid return ValidationResult.Valid
} }
fun findMatchingSourceWithPrecomputedPrefixes(
dictionaryPrefix: String,
sourcesWithPrefixes: List<Pair<DictionarySource, String?>>
): DictionarySource? {
return sourcesWithPrefixes.find { (_, sourcePrefix) ->
sourcePrefix != null && dictionaryPrefix.equals(sourcePrefix, ignoreCase = true)
}?.first
}
} }
@@ -30,18 +30,24 @@ class ManageDictionarySourcesUseCase(
preferencesManager.removeDictionarySource(sourceId) preferencesManager.removeDictionarySource(sourceId)
} }
suspend fun removeSourceForDictionary(dictionary: Dictionary) { suspend fun removeSourceForDictionary(
val dictionaryPrefix = dictionary.name dictionary: Dictionary,
val sources = preferencesManager.dictionarySources.first() remainingDictionaryFileNames: Collection<String>?,
val sourcesWithPrefixes = sources.map { source -> ) {
source to DictionarySource.extractPrefix(source.urlTemplate) if (remainingDictionaryFileNames == null) return
}
val matchingSource = validator.findMatchingSourceWithPrecomputedPrefixes(dictionaryPrefix, sourcesWithPrefixes) val sources = preferencesManager.dictionarySources.first()
if (matchingSource != null) { val sourceIdsToRemove = sources
preferencesManager.removeDictionarySource(matchingSource.id) .filter { source ->
DictionarySourceFileMatcher.matches(source, dictionary) &&
remainingDictionaryFileNames.none { fileName ->
DictionarySourceFileMatcher.matches(source, fileName)
} }
} }
.map(DictionarySource::id)
preferencesManager.removeDictionarySources(sourceIdsToRemove)
}
sealed class AddSourceResult { sealed class AddSourceResult {
object Success : AddSourceResult() object Success : AddSourceResult()
@@ -40,6 +40,10 @@ fun Throwable.isNetworkError(): Boolean {
if (isSslHandshakeError()) return true if (isSslHandshakeError()) return true
return when (this) { return when (this) {
is java.net.UnknownHostException,
is java.net.ConnectException,
is java.net.NoRouteToHostException,
is java.net.BindException -> true
is java.io.IOException -> { is java.io.IOException -> {
val message = message ?: "" val message = message ?: ""
message.contains("connection", ignoreCase = true) || message.contains("connection", ignoreCase = true) ||
@@ -49,9 +53,14 @@ fun Throwable.isNetworkError(): Boolean {
message.contains("unreachable", ignoreCase = true) || message.contains("unreachable", ignoreCase = true) ||
message.contains("no route", ignoreCase = true) || message.contains("no route", ignoreCase = true) ||
message.contains("broken pipe", ignoreCase = true) || message.contains("broken pipe", ignoreCase = true) ||
message.contains("resolve host", ignoreCase = true) ||
message.contains("unexpected end of stream", ignoreCase = true) ||
message.contains("socket closed", ignoreCase = true) ||
message.contains("ECONNRESET", ignoreCase = true) || message.contains("ECONNRESET", ignoreCase = true) ||
message.contains("ECONNREFUSED", ignoreCase = true) || message.contains("ECONNREFUSED", ignoreCase = true) ||
message.contains("ENETUNREACH", ignoreCase = true) message.contains("ECONNABORTED", ignoreCase = true) ||
message.contains("ENETUNREACH", ignoreCase = true) ||
message.contains("EHOSTUNREACH", ignoreCase = true)
} }
else -> false else -> false
} }
@@ -9,6 +9,32 @@ fun String.sanitizeQuery(): String {
.take(MAX_QUERY_LENGTH) .take(MAX_QUERY_LENGTH)
} }
fun String.foldLatinDiacritics(): String {
val decomposed = java.text.Normalizer.normalize(this, java.text.Normalizer.Form.NFD)
val folded = StringBuilder(decomposed.length)
var followsLatinBase = false
var index = 0
while (index < decomposed.length) {
val codePoint = decomposed.codePointAt(index)
val type = Character.getType(codePoint)
val isMark = type == Character.NON_SPACING_MARK.toInt() ||
type == Character.COMBINING_SPACING_MARK.toInt() ||
type == Character.ENCLOSING_MARK.toInt()
if (!(isMark && followsLatinBase)) {
folded.appendCodePoint(codePoint)
}
if (!isMark) {
followsLatinBase = Character.UnicodeScript.of(codePoint) == Character.UnicodeScript.LATIN
}
index += Character.charCount(codePoint)
}
return java.text.Normalizer.normalize(folded, java.text.Normalizer.Form.NFC)
}
fun String.sanitizeCacheKey(): String { fun String.sanitizeCacheKey(): String {
return this.replace(":", "_").replace("\n", "_").replace("\t", "_") return this.replace(":", "_").replace("\n", "_").replace("\t", "_")
} }
@@ -6,6 +6,7 @@ import androidx.datastore.preferences.core.Preferences
import androidx.datastore.preferences.core.edit import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.booleanPreferencesKey import androidx.datastore.preferences.core.booleanPreferencesKey
import androidx.datastore.preferences.core.stringPreferencesKey import androidx.datastore.preferences.core.stringPreferencesKey
import androidx.datastore.preferences.core.stringSetPreferencesKey
import androidx.datastore.preferences.preferencesDataStore import androidx.datastore.preferences.preferencesDataStore
import com.example.research.core.domain.model.DictionarySource import com.example.research.core.domain.model.DictionarySource
import kotlinx.coroutines.flow.Flow import kotlinx.coroutines.flow.Flow
@@ -23,6 +24,8 @@ class PreferencesManager(private val context: Context) {
val IS_THEME_EXPANDED = booleanPreferencesKey("is_theme_expanded") val IS_THEME_EXPANDED = booleanPreferencesKey("is_theme_expanded")
val IS_LANGUAGE_EXPANDED = booleanPreferencesKey("is_language_expanded") val IS_LANGUAGE_EXPANDED = booleanPreferencesKey("is_language_expanded")
val IS_DICTIONARIES_EXPANDED = booleanPreferencesKey("is_dictionaries_expanded") val IS_DICTIONARIES_EXPANDED = booleanPreferencesKey("is_dictionaries_expanded")
val DISABLED_DICTIONARY_PATHS = stringSetPreferencesKey("disabled_dictionary_paths")
val HAD_NO_DICTIONARIES = booleanPreferencesKey("had_no_dictionaries")
} }
private val json = Json { ignoreUnknownKeys = true } private val json = Json { ignoreUnknownKeys = true }
@@ -60,6 +63,44 @@ class PreferencesManager(private val context: Context) {
} }
} }
val disabledDictionaryPaths: Flow<Set<String>> = context.dataStore.data
.map { preferences ->
preferences[PreferencesKeys.DISABLED_DICTIONARY_PATHS] ?: emptySet()
}
val hadNoDictionaries: Flow<Boolean> = context.dataStore.data
.map { preferences ->
preferences[PreferencesKeys.HAD_NO_DICTIONARIES] ?: true
}
suspend fun setHadNoDictionaries(value: Boolean) {
context.dataStore.edit { preferences ->
preferences[PreferencesKeys.HAD_NO_DICTIONARIES] = value
}
}
suspend fun setDictionaryActive(path: String, isActive: Boolean) {
context.dataStore.edit { preferences ->
val disabledPaths = preferences[PreferencesKeys.DISABLED_DICTIONARY_PATHS]
.orEmpty()
.toMutableSet()
if (isActive) {
disabledPaths.remove(path)
} else {
disabledPaths.add(path)
}
if (disabledPaths.isEmpty()) {
preferences.remove(PreferencesKeys.DISABLED_DICTIONARY_PATHS)
} else {
preferences[PreferencesKeys.DISABLED_DICTIONARY_PATHS] = disabledPaths
}
}
}
suspend fun removeDictionaryActiveState(path: String) {
setDictionaryActive(path, true)
}
suspend fun saveThemeMode(theme: String) { suspend fun saveThemeMode(theme: String) {
context.dataStore.edit { preferences -> context.dataStore.edit { preferences ->
preferences[PreferencesKeys.THEME_MODE] = theme preferences[PreferencesKeys.THEME_MODE] = theme
@@ -15,74 +15,49 @@ import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.CancellationException import kotlinx.coroutines.CancellationException
import java.io.File import java.io.File
/**
* Two-phase search paging.
*
* Page one (key = null) is the ranked head: the capped, relevance-ordered result the
* search screen has always shown, merged across active dictionaries. Every range scan
* that hit its cap hands back a [IndexSearcher.RangeCursor]; those become the tail key.
*
* Tail pages continue the unfinished range scans directly from the index files and
* merge them alphabetically (k-way, by index word), so scrolling eventually surfaces
* every prefix match in every active dictionary. Articles already shown -- by the head
* or by an earlier tail page, including diacritic alias duplicates -- are filtered
* through [emittedArticles], which grows only as far as the user actually scrolls.
*
* A new query builds a new PagingSource (see SearchViewModel's flatMapLatest), so an
* in-flight head or tail load is cancelled by Paging when the user keeps typing.
*/
class IndexEntryPagingSource( class IndexEntryPagingSource(
private val indexSearcher: IndexSearcher, private val indexSearcher: IndexSearcher,
private val dictionaries: List<Dictionary>, private val dictionaries: List<Dictionary>,
private val query: String private val query: String
) : PagingSource<Int, IndexEntry>() { ) : PagingSource<IndexEntryPagingSource.TailKey, IndexEntry>() {
private var searchResults: List<IndexEntry>? = null data class StreamCursor(
val dictPosition: Int,
val cursor: IndexSearcher.RangeCursor
)
override val jumpingSupported: Boolean = true data class TailKey(val streams: List<StreamCursor>)
override suspend fun load(params: LoadParams<Int>): LoadResult<Int, IndexEntry> { private data class EmittedKey(
val offset = params.key ?: 0 val dictPosition: Int,
val word: String,
val offset: Long
)
private val emittedArticles = HashSet<EmittedKey>()
private val activeDictionaries = dictionaries.withIndex().filter { it.value.isActive }
override suspend fun load(params: LoadParams<TailKey>): LoadResult<TailKey, IndexEntry> {
return try { return try {
val allResults = searchResults ?: ReSearchTrace.asyncSection(ReSearchTrace.SEARCH_PAGING) { val key = params.key
coroutineScope { if (key == null) loadHead() else loadTail(key, params.loadSize)
val rawResults = dictionaries
.filter { it.isActive }
.map { dict ->
async {
val indexFile = File(dict.indexPath)
if (!indexFile.exists()) return@async emptyList()
try {
val results = indexSearcher.search(
pathOrUri = indexFile.absolutePath,
query = query,
includeSubstringMatches = false
)
results.map {
it.withDictionary(
dict.metadata.name.ifBlank { dict.name },
dict.path
)
}
} catch (e: CancellationException) {
throw e
} catch (_: Exception) {
emptyList()
}
}
}
.awaitAll()
.flatten()
// Precompute sorting keys to optimize comparator performance.
// entry.word is already lowercase from indexing.
val normalizedQuery = query.sanitizeQuery().lowercase().trim()
rawResults.rankBySearchRelevance(
SearchRankingContext(normalizedQuery)
)
}
}.also { searchResults = it }
val totalSize = allResults.size
val startIndex = offset.coerceAtLeast(0).coerceAtMost(totalSize)
val endIndex = (startIndex + params.loadSize).coerceAtMost(totalSize)
val pageItems = allResults.subList(startIndex, endIndex)
val hasMore = endIndex < totalSize
val nextKey = if (hasMore) endIndex else null
val prevKey = if (offset == 0) null else (offset - params.loadSize).coerceAtLeast(0)
LoadResult.Page(
data = pageItems,
prevKey = prevKey,
nextKey = nextKey
)
} catch (e: CancellationException) { } catch (e: CancellationException) {
throw e throw e
} catch (e: Exception) { } catch (e: Exception) {
@@ -90,10 +65,130 @@ class IndexEntryPagingSource(
} }
} }
override fun getRefreshKey(state: PagingState<Int, IndexEntry>): Int? { private suspend fun loadHead(): LoadResult<TailKey, IndexEntry> =
return state.anchorPosition?.let { anchorPosition -> ReSearchTrace.asyncSection(ReSearchTrace.SEARCH_PAGING) {
state.closestPageToPosition(anchorPosition)?.prevKey?.plus(state.config.pageSize) coroutineScope {
?: state.closestPageToPosition(anchorPosition)?.nextKey?.minus(state.config.pageSize) val perDictionary = activeDictionaries
.map { (position, dict) ->
async {
val indexFile = File(dict.indexPath)
if (!indexFile.exists()) return@async null
try {
val result = indexSearcher.searchWithCursors(
pathOrUri = indexFile.absolutePath,
query = query
)
Triple(position, dict, result)
} catch (e: CancellationException) {
throw e
} catch (_: Exception) {
null
} }
} }
}
.awaitAll()
.filterNotNull()
val rawResults = perDictionary.flatMap { (position, dict, result) ->
result.entries.map { entry ->
position to entry.withDictionary(
dict.metadata.name.ifBlank { dict.name },
dict.path
)
}
}
val normalizedQuery = query.sanitizeQuery().lowercase().trim()
val ranked = rawResults.map { it.second }
.rankBySearchRelevance(SearchRankingContext(normalizedQuery))
.distinctBy { Triple(it.dictionaryPath, it.word, it.offset.value) }
rawResults.forEach { (position, entry) -> emittedArticles.add(entry.emittedKey(position)) }
val streams = perDictionary.flatMap { (position, _, result) ->
result.cursors.map { StreamCursor(position, it) }
}
LoadResult.Page(
data = ranked,
prevKey = null,
nextKey = if (streams.isEmpty()) null else TailKey(streams)
)
}
}
private suspend fun loadTail(key: TailKey, loadSize: Int): LoadResult<TailKey, IndexEntry> {
val streams = key.streams.map { TailStream(it.dictPosition, it.cursor) }
val page = ArrayList<IndexEntry>(loadSize)
val pageKeys = HashSet<EmittedKey>()
val chunkSize = maxOf(loadSize, MIN_CHUNK)
while (page.size < loadSize) {
streams.forEach { it.ensureBuffered(chunkSize) }
val next = streams
.filter { it.hasBuffered }
.minByOrNull { it.peekWord }
?: break
val scanned = next.consume()
val dict = dictionaries[next.dictPosition]
val entry = scanned.entry.withDictionary(
dict.metadata.name.ifBlank { dict.name },
dict.path
)
val emittedKey = entry.emittedKey(next.dictPosition)
if (emittedKey !in emittedArticles && pageKeys.add(emittedKey)) {
page.add(entry)
}
}
emittedArticles.addAll(pageKeys)
val remaining = streams.mapNotNull { stream ->
stream.nextCursor?.let { StreamCursor(stream.dictPosition, it) }
}
return LoadResult.Page(
data = page,
prevKey = null,
nextKey = if (remaining.isEmpty()) null else TailKey(remaining)
)
}
private inner class TailStream(
val dictPosition: Int,
initialCursor: IndexSearcher.RangeCursor
) {
var nextCursor: IndexSearcher.RangeCursor? = initialCursor
private set
private var buffer: List<IndexSearcher.ScannedEntry> = emptyList()
private var position = 0
val hasBuffered: Boolean get() = position < buffer.size
val peekWord: String get() = buffer[position].entry.word
suspend fun ensureBuffered(chunkSize: Int) {
if (hasBuffered) return
val cursor = nextCursor ?: return
val indexPath = File(dictionaries[dictPosition].indexPath).absolutePath
buffer = indexSearcher.scanTailChunk(indexPath, cursor, chunkSize)
position = 0
if (buffer.isEmpty()) nextCursor = null
}
fun consume(): IndexSearcher.ScannedEntry {
val scanned = buffer[position]
position++
nextCursor = scanned.cursorAfter
return scanned
}
}
private fun IndexEntry.emittedKey(dictPosition: Int) =
EmittedKey(dictPosition, word, offset.value)
override fun getRefreshKey(state: PagingState<TailKey, IndexEntry>): TailKey? {
return null
}
} }
private const val MIN_CHUNK = 48
@@ -58,17 +58,23 @@ object DslCharsetDetector {
private fun looksLikeUtf8(inputStream: java.io.InputStream): Boolean { private fun looksLikeUtf8(inputStream: java.io.InputStream): Boolean {
if (!inputStream.markSupported()) return false if (!inputStream.markSupported()) return false
inputStream.mark(8192) inputStream.mark(8192)
return try { try {
val sample = ByteArray(8192) val sample = ByteArray(8192)
val read = inputStream.read(sample) val read = inputStream.read(sample)
if (read <= 0) return false if (read <= 0) return false
val decoder = Charsets.UTF_8.newDecoder() val decoder = Charsets.UTF_8.newDecoder()
.onMalformedInput(java.nio.charset.CodingErrorAction.REPORT) .onMalformedInput(java.nio.charset.CodingErrorAction.REPORT)
.onUnmappableCharacter(java.nio.charset.CodingErrorAction.REPORT) .onUnmappableCharacter(java.nio.charset.CodingErrorAction.REPORT)
decoder.decode(java.nio.ByteBuffer.wrap(sample, 0, read)) val byteBuffer = java.nio.ByteBuffer.wrap(sample, 0, read)
true val charBuffer = java.nio.CharBuffer.allocate(8192)
while (true) {
val result = decoder.decode(byteBuffer, charBuffer, false)
if (result.isError) return false
if (result.isUnderflow) return true
charBuffer.clear()
}
} catch (_: Exception) { } catch (_: Exception) {
false return false
} finally { } finally {
inputStream.reset() inputStream.reset()
} }
@@ -24,10 +24,11 @@ object DslHeadwordParser {
} }
if (trimmed.indexOf('{') == -1 && trimmed.indexOf('[') == -1) { if (trimmed.indexOf('{') == -1 && trimmed.indexOf('[') == -1) {
val unescaped = unescape(trimmed)
return ParsedHeadword( return ParsedHeadword(
simplified = trimmed, simplified = unescaped,
displayText = trimmed, displayText = unescaped,
searchableText = trimmed.lowercase() searchableText = unescaped.lowercase()
) )
} }
@@ -60,9 +61,9 @@ object DslHeadwordParser {
val searchableText = createSearchableText(trimmed) val searchableText = createSearchableText(trimmed)
return ParsedHeadword( return ParsedHeadword(
simplified = simplified, simplified = unescape(simplified),
displayText = displayText, displayText = unescape(displayText),
searchableText = searchableText searchableText = unescape(searchableText)
) )
} }
@@ -129,8 +130,17 @@ object DslHeadwordParser {
if (value.indexOf('{') == -1 && value.indexOf('}') == -1) return value if (value.indexOf('{') == -1 && value.indexOf('}') == -1) return value
return buildString(value.length) { return buildString(value.length) {
value.forEach { char -> var index = 0
while (index < value.length) {
val char = value[index]
if (char == '\\' && index + 1 < value.length) {
append(char)
append(value[index + 1])
index += 2
} else {
if (char != '{' && char != '}') append(char) if (char != '{' && char != '}') append(char)
index++
}
} }
} }
} }
@@ -138,6 +148,10 @@ object DslHeadwordParser {
private fun firstFormattingTagIndex(value: String): Int { private fun firstFormattingTagIndex(value: String): Int {
var index = 0 var index = 0
while (index < value.length) { while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
continue
}
if (value[index] == '[' && formattingTagEnd(value, index) > index) { if (value[index] == '[' && formattingTagEnd(value, index) > index) {
return index return index
} }
@@ -149,8 +163,12 @@ object DslHeadwordParser {
private fun firstCurlyContent(value: String): String? { private fun firstCurlyContent(value: String): String? {
var index = 0 var index = 0
while (index < value.length) { while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
continue
}
if (value[index] == '{') { if (value[index] == '{') {
val end = value.indexOf('}', startIndex = index + 1) val end = matchingCurlyEnd(value, index + 1)
if (end > index + 1) { if (end > index + 1) {
return value.substring(index + 1, end) return value.substring(index + 1, end)
} }
@@ -161,7 +179,7 @@ object DslHeadwordParser {
} }
private fun substringBeforeFirstBracket(value: String): String { private fun substringBeforeFirstBracket(value: String): String {
val bracketIndex = value.indexOf('[') val bracketIndex = indexOfUnescaped(value, '[')
return if (bracketIndex >= 0) value.substring(0, bracketIndex) else value return if (bracketIndex >= 0) value.substring(0, bracketIndex) else value
} }
@@ -171,8 +189,12 @@ object DslHeadwordParser {
var index = 0 var index = 0
while (index < value.length) { while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
continue
}
if (value[index] == '{') { if (value[index] == '{') {
val end = value.indexOf('}', startIndex = index + 1) val end = matchingCurlyEnd(value, index + 1)
if (end >= 0 && (removeEmpty || end > index + 1)) { if (end >= 0 && (removeEmpty || end > index + 1)) {
if (builder == null) { if (builder == null) {
builder = StringBuilder(value.length) builder = StringBuilder(value.length)
@@ -197,6 +219,10 @@ object DslHeadwordParser {
var index = 0 var index = 0
while (index < value.length) { while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
continue
}
if (value[index] == '[') { if (value[index] == '[') {
val tagEnd = formattingTagEnd(value, index) val tagEnd = formattingTagEnd(value, index)
if (tagEnd > index) { if (tagEnd > index) {
@@ -217,6 +243,56 @@ object DslHeadwordParser {
}?.toString() ?: value }?.toString() ?: value
} }
private fun matchingCurlyEnd(value: String, startIndex: Int): Int {
var index = startIndex
while (index < value.length) {
when {
value[index] == '\\' && index + 1 < value.length -> index += 2
value[index] == '{' -> return -1
value[index] == '}' -> return index
else -> index++
}
}
return -1
}
private fun indexOfUnescaped(
value: String,
target: Char,
startIndex: Int = 0,
): Int {
var index = startIndex.coerceAtLeast(0)
while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
} else if (value[index] == target) {
return index
} else {
index++
}
}
return -1
}
private fun unescape(value: String): String {
val firstEscape = value.indexOf('\\')
if (firstEscape < 0) return value
return buildString(value.length) {
append(value, 0, firstEscape)
var index = firstEscape
while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
append(value[index + 1])
index += 2
} else {
append(value[index])
index++
}
}
}
}
private fun formattingTagEnd(value: String, openBracketIndex: Int): Int { private fun formattingTagEnd(value: String, openBracketIndex: Int): Int {
val tokenStart = openBracketIndex + 1 val tokenStart = openBracketIndex + 1
if (tokenStart >= value.length) return -1 if (tokenStart >= value.length) return -1
@@ -4,6 +4,7 @@ import android.util.Log
import com.example.research.core.domain.model.ArticleLength import com.example.research.core.domain.model.ArticleLength
import com.example.research.core.domain.model.ArticleOffset import com.example.research.core.domain.model.ArticleOffset
import com.example.research.core.domain.model.IndexEntry import com.example.research.core.domain.model.IndexEntry
import com.example.research.core.util.foldLatinDiacritics
import com.example.research.core.util.removeAccentTagsForIndexing import com.example.research.core.util.removeAccentTagsForIndexing
import kotlinx.coroutines.Deferred import kotlinx.coroutines.Deferred
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
@@ -23,7 +24,7 @@ class DslIndexer(context: Context? = null) {
companion object { companion object {
private const val TAG = "DslIndexer" private const val TAG = "DslIndexer"
const val INDEX_VERSION = 4 const val INDEX_VERSION = 6
const val BUFFER_SIZE = 1048576 const val BUFFER_SIZE = 1048576
const val BATCH_BUFFER_SIZE = 524288 const val BATCH_BUFFER_SIZE = 524288
private const val INDEX_SORT_PARALLELISM = 1 private const val INDEX_SORT_PARALLELISM = 1
@@ -40,6 +41,9 @@ class DslIndexer(context: Context? = null) {
private const val IN_MEMORY_SORT_MAX_ENTRIES = 140_000 private const val IN_MEMORY_SORT_MAX_ENTRIES = 140_000
private const val ESTIMATED_IN_MEMORY_ENTRY_BYTES = 768L private const val ESTIMATED_IN_MEMORY_ENTRY_BYTES = 768L
private const val IN_MEMORY_SORT_HEAP_RESERVE_BYTES = 96L * 1024L * 1024L private const val IN_MEMORY_SORT_HEAP_RESERVE_BYTES = 96L * 1024L * 1024L
fun foldedIndexFile(indexFile: File): File =
File(indexFile.parent ?: ".", "${indexFile.name}.fold")
} }
suspend fun createIndex( suspend fun createIndex(
@@ -56,21 +60,25 @@ class DslIndexer(context: Context? = null) {
java.io.BufferedInputStream(dslInputStream) java.io.BufferedInputStream(dslInputStream)
} }
val tempFile = File(indexFile.parent, "${indexFile.name}.tmp") val tempFile = File(indexFile.parent, "${indexFile.name}.tmp")
val entryCount: Int val counts: ParsedCounts
try { try {
try { foldedIndexFile(indexFile).delete() } catch (_: Throwable) { /* Legacy v5 leftover */ }
onOperationChange("Parsing file") onOperationChange("Parsing file")
onFileProgress(0.0f) onFileProgress(0.0f)
entryCount = parseDslToTempFile(bufferedStream, tempFile, counts = parseDslToTempFile(bufferedStream, tempFile,
onProgress, onFileProgress, onCharsetDetected) onProgress, onFileProgress, onCharsetDetected)
onFileProgress(0.5f) onFileProgress(0.5f)
onOperationChange("Sorting index") onOperationChange("Sorting index")
sortAndWriteIndexFile(tempFile, indexFile, entryCount, onOperationChange, onFileProgress) sortAndWriteIndexFile(tempFile, indexFile, counts, onOperationChange, onFileProgress)
onFileProgress(1f)
} finally { } finally {
try { tempFile.delete() } catch (_: Throwable) { /* Ignore cleanup failure */ } try { tempFile.delete() } catch (_: Throwable) { /* Ignore cleanup failure */ }
} }
try { onProgress(entryCount) } catch (_: Throwable) { /* Ignore progress callback failure */ } try { onProgress(counts.articleCount) } catch (_: Throwable) { /* Ignore progress callback failure */ }
return entryCount return counts.articleCount
} }
private data class ParsedCounts(val articleCount: Int, val totalEntries: Int)
// Blocking file I/O is safe across this file: every suspend fun here is reached only // Blocking file I/O is safe across this file: every suspend fun here is reached only
// from createIndex(), which the sole caller (KotlinDictionaryEngine) runs inside // from createIndex(), which the sole caller (KotlinDictionaryEngine) runs inside
// withContext(Dispatchers.IO). The dispatcher is invisible across the suspend-call // withContext(Dispatchers.IO). The dispatcher is invisible across the suspend-call
@@ -83,7 +91,7 @@ class DslIndexer(context: Context? = null) {
onProgress: (Int) -> Unit, onProgress: (Int) -> Unit,
onFileProgress: (Float) -> Unit = {}, onFileProgress: (Float) -> Unit = {},
onCharsetDetected: (DslCharsetDetector.DetectedCharset) -> Unit = {} onCharsetDetected: (DslCharsetDetector.DetectedCharset) -> Unit = {}
): Int { ): ParsedCounts {
val detectedCharset = DslCharsetDetector.detect(inputStream) val detectedCharset = DslCharsetDetector.detect(inputStream)
onCharsetDetected(detectedCharset) onCharsetDetected(detectedCharset)
@@ -98,6 +106,7 @@ class DslIndexer(context: Context? = null) {
) )
var entryCount = 0 var entryCount = 0
var aliasCount = 0
var lineCount = 0 var lineCount = 0
var headwordLineCount = 0 var headwordLineCount = 0
var bodyLineCount = 0 var bodyLineCount = 0
@@ -127,9 +136,10 @@ class DslIndexer(context: Context? = null) {
fun normalizeSearchableWord(searchableText: String, fallback: String): String { fun normalizeSearchableWord(searchableText: String, fallback: String): String {
// Parser already lowercases and strips accent tags for non-empty results; // Parser already lowercases and strips accent tags for non-empty results;
// only the raw-headword fallback still needs normalization. // only the raw-headword fallback still needs normalization.
return searchableText.ifEmpty { val normalized = searchableText.ifEmpty {
fallback.removeAccentTagsForIndexing().lowercase() fallback.removeAccentTagsForIndexing().lowercase()
} }
return java.text.Normalizer.normalize(normalized, java.text.Normalizer.Form.NFC)
} }
DataOutputStream(BufferedOutputStream(FileOutputStream(tempFile), BUFFER_SIZE)).use { output -> DataOutputStream(BufferedOutputStream(FileOutputStream(tempFile), BUFFER_SIZE)).use { output ->
@@ -144,12 +154,23 @@ class DslIndexer(context: Context? = null) {
return return
} }
val length = (groupEndOffset - groupStartOffset).coerceAtLeast(0L).toInt() val length = (groupEndOffset - groupStartOffset).coerceAtLeast(0L).toInt()
for (idx in groupSearchable.indices) { fun writeTempEntry(word: String, display: String, isAlias: Boolean) {
writer.write(groupSearchable[idx]) writer.write(word)
writer.write(groupDisplay[idx]) writer.write(display)
output.writeLong(groupStartOffset) output.writeLong(groupStartOffset)
output.writeInt(length) output.writeInt(length)
output.writeByte(if (isAlias) ENTRY_FLAG_ALIAS else 0)
}
for (idx in groupSearchable.indices) {
val searchableWord = groupSearchable[idx]
val displayWord = groupDisplay[idx]
writeTempEntry(searchableWord, displayWord, isAlias = false)
entryCount++ entryCount++
val foldedWord = searchableWord.foldLatinDiacritics()
if (foldedWord != searchableWord) {
writeTempEntry(foldedWord, displayWord, isAlias = true)
aliasCount++
}
} }
groupSearchable.clear() groupSearchable.clear()
groupDisplay.clear() groupDisplay.clear()
@@ -230,26 +251,28 @@ class DslIndexer(context: Context? = null) {
try { onProgress(entryCount) } catch (_: Throwable) { /* Ignore progress callback failure */ } try { onProgress(entryCount) } catch (_: Throwable) { /* Ignore progress callback failure */ }
} }
onFileProgress(0.5f) onFileProgress(0.5f)
return entryCount return ParsedCounts(articleCount = entryCount, totalEntries = entryCount + aliasCount)
} }
// Blocking file I/O safe: reached only from createIndex() (runs on Dispatchers.IO). See parseDslToTempFile(). // Blocking file I/O safe: reached only from createIndex() (runs on Dispatchers.IO). See parseDslToTempFile().
@Suppress("BlockingMethodInNonBlockingContext") @Suppress("BlockingMethodInNonBlockingContext")
private suspend fun sortAndWriteIndexFile(tempFile: File, indexFile: File, entryCount: Int, onOperationChange: (String) -> Unit = {}, onFileProgress: (Float) -> Unit = {}) { private suspend fun sortAndWriteIndexFile(tempFile: File, indexFile: File, counts: ParsedCounts, onOperationChange: (String) -> Unit = {}, onFileProgress: (Float) -> Unit = {}) {
if (entryCount == 0) { if (counts.totalEntries == 0) {
DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile))).use { output -> DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile))).use { output ->
output.writeInt(INDEX_VERSION) output.writeInt(INDEX_VERSION)
output.writeInt(0) output.writeInt(0)
output.writeInt(0) output.writeInt(0)
output.writeInt(0)
} }
return return
} }
if (shouldUseInMemorySort(entryCount)) { if (shouldUseInMemorySort(counts.totalEntries)) {
inMemorySortAndWrite(tempFile, indexFile, entryCount) inMemorySortAndWrite(tempFile, indexFile, counts)
onFileProgress(FINAL_INDEX_DONE_PROGRESS) onFileProgress(FINAL_INDEX_DONE_PROGRESS)
} else { } else {
externalSortAndWrite(tempFile, indexFile, entryCount, onOperationChange, onFileProgress) externalSortAndWrite(tempFile, indexFile, counts, onOperationChange, onFileProgress)
} }
} }
private fun shouldUseInMemorySort(entryCount: Int): Boolean { private fun shouldUseInMemorySort(entryCount: Int): Boolean {
if (entryCount <= IN_MEMORY_SORT_ALWAYS_ENTRIES) return true if (entryCount <= IN_MEMORY_SORT_ALWAYS_ENTRIES) return true
if (entryCount > IN_MEMORY_SORT_MAX_ENTRIES) return false if (entryCount > IN_MEMORY_SORT_MAX_ENTRIES) return false
@@ -264,30 +287,22 @@ class DslIndexer(context: Context? = null) {
} }
// Blocking file I/O safe: reached only from createIndex() (runs on Dispatchers.IO). See parseDslToTempFile(). // Blocking file I/O safe: reached only from createIndex() (runs on Dispatchers.IO). See parseDslToTempFile().
@Suppress("BlockingMethodInNonBlockingContext") @Suppress("BlockingMethodInNonBlockingContext")
private suspend fun inMemorySortAndWrite(tempFile: File, indexFile: File, entryCount: Int) { private suspend fun inMemorySortAndWrite(tempFile: File, indexFile: File, counts: ParsedCounts) {
val entries = ArrayList<IndexEntry>(entryCount) val entries = ArrayList<IndexEntry>(counts.totalEntries)
java.io.DataInputStream(java.io.BufferedInputStream(java.io.FileInputStream(tempFile), BATCH_BUFFER_SIZE)).use { input -> java.io.DataInputStream(java.io.BufferedInputStream(java.io.FileInputStream(tempFile), BATCH_BUFFER_SIZE)).use { input ->
repeat(entryCount) { repeat(counts.totalEntries) {
val word = input.readUtf8String() entries.add(input.readTempEntry())
val originalWord = input.readUtf8String()
val offset = input.readLong()
val length = input.readInt()
entries.add(IndexEntry(
word = word,
originalWord = originalWord,
offset = ArticleOffset(offset),
length = ArticleLength(length)
))
if (entries.size % 5000 == 0) { if (entries.size % 5000 == 0) {
try { yield() } catch (_: Throwable) { /* Ignore yield failure */ } try { yield() } catch (_: Throwable) { /* Ignore yield failure */ }
} }
} }
} }
entries.sort() entries.sort()
writeIndexFile(indexFile, entries) writeIndexFile(indexFile, entries, counts.articleCount)
entries.clear() entries.clear()
} }
private suspend fun externalSortAndWrite(tempFile: File, indexFile: File, entryCount: Int, onOperationChange: (String) -> Unit, onFileProgress: (Float) -> Unit) { private suspend fun externalSortAndWrite(tempFile: File, indexFile: File, counts: ParsedCounts, onOperationChange: (String) -> Unit, onFileProgress: (Float) -> Unit) {
val entryCount = counts.totalEntries
val adaptiveConfig = resourceManager?.calculateAdaptiveConfig(estimatedEntryCount = entryCount.toLong()) val adaptiveConfig = resourceManager?.calculateAdaptiveConfig(estimatedEntryCount = entryCount.toLong())
?: ResourceManager.AdaptiveConfig( ?: ResourceManager.AdaptiveConfig(
batchSize = 100_000, batchSize = 100_000,
@@ -317,16 +332,7 @@ class DslIndexer(context: Context? = null) {
val batch = ArrayList<IndexEntry>(currentBatchSize) val batch = ArrayList<IndexEntry>(currentBatchSize)
val batchEnd = minOf(processed + currentBatchSize, entryCount) val batchEnd = minOf(processed + currentBatchSize, entryCount)
while (processed < batchEnd) { while (processed < batchEnd) {
val word = input.readUtf8String() batch.add(input.readTempEntry())
val originalWord = input.readUtf8String()
val offset = input.readLong()
val length = input.readInt()
batch.add(IndexEntry(
word = word,
originalWord = originalWord,
offset = ArticleOffset(offset),
length = ArticleLength(length)
))
processed++ processed++
} }
@@ -422,7 +428,7 @@ class DslIndexer(context: Context? = null) {
onFileProgress(MERGE_START_PROGRESS) onFileProgress(MERGE_START_PROGRESS)
onOperationChange("Merging ${sortedBatches.size} batches") onOperationChange("Merging ${sortedBatches.size} batches")
mergeSortedBatches(sortedBatches, indexFile, entryCount, onOperationChange, onFileProgress) mergeSortedBatches(sortedBatches, indexFile, counts, onOperationChange, onFileProgress)
onFileProgress(FINAL_INDEX_DONE_PROGRESS) onFileProgress(FINAL_INDEX_DONE_PROGRESS)
} finally { } finally {
sortedBatches.forEach { try { it.delete() } catch (_: Throwable) { /* Ignore cleanup failure */ } } sortedBatches.forEach { try { it.delete() } catch (_: Throwable) { /* Ignore cleanup failure */ } }
@@ -443,6 +449,7 @@ class DslIndexer(context: Context? = null) {
writer.write(entry.originalWord) writer.write(entry.originalWord)
output.writeLong(entry.offset.value) output.writeLong(entry.offset.value)
output.writeInt(entry.length.value) output.writeInt(entry.length.value)
output.writeByte(if (entry.isAlias) ENTRY_FLAG_ALIAS else 0)
} }
} }
} }
@@ -453,15 +460,16 @@ class DslIndexer(context: Context? = null) {
private suspend fun mergeSortedBatches( private suspend fun mergeSortedBatches(
batchFiles: List<File>, batchFiles: List<File>,
indexFile: File, indexFile: File,
totalEntries: Int, counts: ParsedCounts,
onOperationChange: (String) -> Unit = {}, onOperationChange: (String) -> Unit = {},
onFileProgress: (Float) -> Unit, onFileProgress: (Float) -> Unit,
) { ) {
if (batchFiles.isEmpty()) return if (batchFiles.isEmpty()) return
val totalEntries = counts.totalEntries
val sparsePoints = mutableListOf<SparsePoint>() val sparsePoints = mutableListOf<SparsePoint>()
val prefixCounts = HashMap<String, Int>(minOf(totalEntries / 10, 100_000)) val prefixCounts = HashMap<String, Int>(minOf(totalEntries / 10, 100_000))
val headerSize = 4 + 4 + 4 val headerSize = 4 + 4 + 4 + 4
val dataScratchFile = File(indexFile.parent, "${indexFile.name}.data") val dataScratchFile = File(indexFile.parent, "${indexFile.name}.data")
val pq = PriorityQueue<BatchReader>() val pq = PriorityQueue<BatchReader>()
@@ -508,7 +516,8 @@ class DslIndexer(context: Context? = null) {
val origSize = writer.write(entry.originalWord) val origSize = writer.write(entry.originalWord)
dataOutput.writeLong(entry.offset.value) dataOutput.writeLong(entry.offset.value)
dataOutput.writeInt(entry.length.value) dataOutput.writeInt(entry.length.value)
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 dataOutput.writeByte(if (entry.isAlias) ENTRY_FLAG_ALIAS else 0)
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 + 1
entryIndex++ entryIndex++
@@ -542,6 +551,7 @@ class DslIndexer(context: Context? = null) {
DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile), BUFFER_SIZE)).use { output -> DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile), BUFFER_SIZE)).use { output ->
output.writeInt(INDEX_VERSION) output.writeInt(INDEX_VERSION)
output.writeInt(totalEntries) output.writeInt(totalEntries)
output.writeInt(counts.articleCount)
output.writeInt(sparsePoints.size) output.writeInt(sparsePoints.size)
sparsePoints.forEachIndexed { idx, point -> sparsePoints.forEachIndexed { idx, point ->
@@ -632,18 +642,14 @@ class DslIndexer(context: Context? = null) {
} }
private fun readNextEntry(input: java.io.DataInputStream): IndexEntry? { private fun readNextEntry(input: java.io.DataInputStream): IndexEntry? {
return try { return try {
val word = input.readUtf8String() input.readTempEntry()
val originalWord = input.readUtf8String()
val offset = ArticleOffset(input.readLong())
val length = ArticleLength(input.readInt())
IndexEntry(word, originalWord, offset, length)
} catch (_: java.io.EOFException) { } catch (_: java.io.EOFException) {
null null
} }
} }
private fun writeIndexFile(indexFile: File, entries: List<IndexEntry>) { private fun writeIndexFile(indexFile: File, entries: List<IndexEntry>, articleCount: Int) {
val sparseData = buildAdaptiveSparseIndex(entries) val sparseData = buildAdaptiveSparseIndex(entries)
val headerSize = 4 + 4 + 4 val headerSize = 4 + 4 + 4 + 4
// Cache sparse table bytes to avoid redundant conversions // Cache sparse table bytes to avoid redundant conversions
val sparseTableBytes = sparseData.map { CachedBytes.from(it.word) } val sparseTableBytes = sparseData.map { CachedBytes.from(it.word) }
@@ -657,13 +663,14 @@ class DslIndexer(context: Context? = null) {
// Calculate sizes without allocating byte arrays // Calculate sizes without allocating byte arrays
val wordSize = utf8ByteSize(entry.word) val wordSize = utf8ByteSize(entry.word)
val origSize = utf8ByteSize(entry.originalWord) val origSize = utf8ByteSize(entry.originalWord)
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 + 1
} }
DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile))).use { output -> DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile))).use { output ->
val writer = Utf8Writer(output) val writer = Utf8Writer(output)
output.writeInt(INDEX_VERSION) output.writeInt(INDEX_VERSION)
output.writeInt(entries.size) output.writeInt(entries.size)
output.writeInt(articleCount)
output.writeInt(sparseData.size) output.writeInt(sparseData.size)
sparseData.forEach { point -> sparseData.forEach { point ->
@@ -677,6 +684,7 @@ class DslIndexer(context: Context? = null) {
writer.write(entry.originalWord) writer.write(entry.originalWord)
output.writeLong(entry.offset.value) output.writeLong(entry.offset.value)
output.writeInt(entry.length.value) output.writeInt(entry.length.value)
output.writeByte(if (entry.isAlias) ENTRY_FLAG_ALIAS else 0)
} }
} }
} }
@@ -707,7 +715,7 @@ class DslIndexer(context: Context? = null) {
// Calculate sizes without allocating byte arrays // Calculate sizes without allocating byte arrays
val wordSize = utf8ByteSize(entry.word) val wordSize = utf8ByteSize(entry.word)
val origSize = utf8ByteSize(entry.originalWord) val origSize = utf8ByteSize(entry.originalWord)
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 + 1
} }
return sparsePoints return sparsePoints
@@ -824,6 +832,24 @@ private fun java.io.DataInputStream.readUtf8String(): String {
readFully(bytes) readFully(bytes)
return String(bytes, Charsets.UTF_8) return String(bytes, Charsets.UTF_8)
} }
internal const val ENTRY_FLAG_ALIAS = 1
private fun java.io.DataInputStream.readTempEntry(): IndexEntry {
val word = readUtf8String()
val originalWord = readUtf8String()
val offset = readLong()
val length = readInt()
val flags = readUnsignedByte()
return IndexEntry(
word = word,
originalWord = originalWord,
offset = ArticleOffset(offset),
length = ArticleLength(length),
isAlias = flags and ENTRY_FLAG_ALIAS != 0
)
}
private enum class LineKind { private enum class LineKind {
HEADWORD, HEADWORD,
BODY, BODY,
@@ -5,6 +5,7 @@ import com.example.research.R
import com.example.research.core.domain.model.ArticleLength import com.example.research.core.domain.model.ArticleLength
import com.example.research.core.domain.model.ArticleOffset import com.example.research.core.domain.model.ArticleOffset
import com.example.research.core.domain.model.IndexEntry import com.example.research.core.domain.model.IndexEntry
import com.example.research.core.util.foldLatinDiacritics
import com.example.research.core.util.DictionaryError import com.example.research.core.util.DictionaryError
import com.example.research.core.util.sanitizeCacheKey import com.example.research.core.util.sanitizeCacheKey
import com.example.research.core.util.sanitizeQuery import com.example.research.core.util.sanitizeQuery
@@ -23,7 +24,8 @@ import java.io.RandomAccessFile
class IndexSearcher(private val context: Context) { class IndexSearcher(private val context: Context) {
private val metadataCache = MetadataLruCache(maxBytes = 64L * 1024L * 1024L) private val metadataCache = MetadataLruCache(maxBytes = 64L * 1024L * 1024L)
private val resultsCache = ResultsLruCache(maxEntries = 20) private val resultsCache = ResultsLruCache<List<IndexEntry>>(maxEntries = 20)
private val cursorResultsCache = ResultsLruCache<CursorSearchResult>(maxEntries = 20)
/** /**
* Pre-allocated reusable buffer for readEntry() to reduce allocations. * Pre-allocated reusable buffer for readEntry() to reduce allocations.
@@ -43,13 +45,26 @@ class IndexSearcher(private val context: Context) {
val dataStartOffset: Long val dataStartOffset: Long
) )
data class IndexComparisonSummary( data class RangeCursor(
val expectedCount: Int, val rangeQuery: String,
val actualCount: Int, val absoluteIndex: Int,
val comparedCount: Int val fileOffset: Long
) )
class IndexComparisonException(message: String) : IllegalStateException(message) internal class RangeScanResult(
val entries: List<IndexEntry>,
val nextCursor: RangeCursor?
)
class CursorSearchResult(
val entries: List<IndexEntry>,
val cursors: List<RangeCursor>
)
class ScannedEntry(
val entry: IndexEntry,
val cursorAfter: RangeCursor?
)
suspend fun findFirstEntry(pathOrUri: String, query: String): IndexEntry? = withContext(Dispatchers.Default) { suspend fun findFirstEntry(pathOrUri: String, query: String): IndexEntry? = withContext(Dispatchers.Default) {
val normalizedQuery = query.sanitizeQuery().lowercase().trim() val normalizedQuery = query.sanitizeQuery().lowercase().trim()
@@ -66,7 +81,7 @@ class IndexSearcher(private val context: Context) {
var iterationCount = 0 var iterationCount = 0
while (currentIdx < count) { while (currentIdx < count) {
if (iterationCount % 1000 == 0) yield() if (iterationCount % 1000 == 0) yield()
val nextEntry = readEntry(source) val nextEntry = readEntry(source, metadata)
val nextWordLower = nextEntry.word.lowercase() val nextWordLower = nextEntry.word.lowercase()
if (!nextWordLower.startsWith(normalizedQuery)) break if (!nextWordLower.startsWith(normalizedQuery)) break
if (nextWordLower == normalizedQuery) return@withContext nextEntry.copy(word = nextWordLower) if (nextWordLower == normalizedQuery) return@withContext nextEntry.copy(word = nextWordLower)
@@ -93,6 +108,7 @@ class IndexSearcher(private val context: Context) {
return@withContext emptyList() return@withContext emptyList()
} }
val normalizedQuery = query.sanitizeQuery().lowercase().trim() val normalizedQuery = query.sanitizeQuery().lowercase().trim()
val foldedQuery = normalizedQuery.foldLatinDiacritics()
val cacheKey = "$pathOrUri:${normalizedQuery.sanitizeCacheKey()}" val cacheKey = "$pathOrUri:${normalizedQuery.sanitizeCacheKey()}"
resultsCache.get(cacheKey)?.let { resultsCache.get(cacheKey)?.let {
return@withContext it return@withContext it
@@ -102,33 +118,34 @@ class IndexSearcher(private val context: Context) {
val count = metadata.totalCount val count = metadata.totalCount
if (count == 0) return@withContext emptyList() if (count == 0) return@withContext emptyList()
val results = openSource(pathOrUri).use { source -> val results = openSource(pathOrUri).use { source ->
val prefixMatches = mutableListOf<IndexEntry>() val prefixMatches = readPrefixMatches(source, metadata, normalizedQuery).entries.toMutableList()
val substringMatches = mutableListOf<IndexEntry>() if (foldedQuery != normalizedQuery) {
val firstMatch = findFirstMatch(source, metadata, normalizedQuery) prefixMatches.addAll(readPrefixMatches(source, metadata, foldedQuery).entries)
if (firstMatch != null) {
prefixMatches.addAll(readMatchingEntries(source, count, firstMatch, normalizedQuery))
} }
val substringMatches = mutableListOf<IndexEntry>()
val seenArticles = prefixMatches.mapTo(HashSet()) { it.articleKey() }
if (includeSubstringMatches && if (includeSubstringMatches &&
count <= SUBSTRING_SCAN_MAX_ENTRIES && count <= SUBSTRING_SCAN_MAX_ENTRIES &&
prefixMatches.size < 100 && seenArticles.size < MAX_SUBSTRING_RESULTS &&
normalizedQuery.length >= 2 normalizedQuery.length >= 2
) { ) {
val prefixMatchWords = prefixMatches.map { it.word }.toSet()
source.seek(metadata.dataStartOffset) source.seek(metadata.dataStartOffset)
for (i in 0 until count) { for (i in 0 until count) {
if (i % 1000 == 0) yield() if (i % 1000 == 0) yield()
if (substringMatches.size >= 100) break if (substringMatches.size >= MAX_SUBSTRING_RESULTS) break
val entry = readEntry(source) val entry = readEntry(source, metadata)
val entryWordLower = entry.word.lowercase() val entryWordLower = entry.word.lowercase()
if (entryWordLower.contains(normalizedQuery) && entryWordLower !in prefixMatchWords) { if (entryWordLower.contains(normalizedQuery) &&
seenArticles.add(entry.articleKey())
) {
substringMatches.add(entry.copy(word = entryWordLower)) substringMatches.add(entry.copy(word = entryWordLower))
} }
} }
} }
val combinedResults = prefixMatches + substringMatches (prefixMatches + substringMatches)
combinedResults.rankBySearchRelevance( .rankBySearchRelevance(SearchRankingContext(normalizedQuery, foldedQuery))
SearchRankingContext(normalizedQuery) .distinctBy { it.articleKey() }
)
} }
resultsCache.put(cacheKey, results) resultsCache.put(cacheKey, results)
results results
@@ -148,97 +165,81 @@ class IndexSearcher(private val context: Context) {
metadataCache.getOrPut(pathOrUri) { loadMetadata(pathOrUri) } metadataCache.getOrPut(pathOrUri) { loadMetadata(pathOrUri) }
} }
suspend fun compareEntryGroups( suspend fun searchWithCursors(
expectedIndexPath: String, pathOrUri: String,
actualIndexPath: String, query: String
onGroup: suspend (word: String, expected: List<IndexEntry>, actual: List<IndexEntry>) -> Unit ): CursorSearchResult = withContext(Dispatchers.Default) {
): IndexComparisonSummary = withContext(Dispatchers.IO) { if (query.isBlank()) {
val expectedMetadata = loadMetadata(expectedIndexPath) return@withContext CursorSearchResult(emptyList(), emptyList())
val actualMetadata = loadMetadata(actualIndexPath)
if (expectedMetadata.totalCount != actualMetadata.totalCount) {
throw IndexComparisonException(
"Index entry count differs: expected=${expectedMetadata.totalCount}, " +
"actual=${actualMetadata.totalCount}"
)
} }
val normalizedQuery = query.sanitizeQuery().lowercase().trim()
openSource(expectedIndexPath).use { expectedSource -> val foldedQuery = normalizedQuery.foldLatinDiacritics()
openSource(actualIndexPath).use { actualSource -> val cacheKey = "$pathOrUri:${normalizedQuery.sanitizeCacheKey()}"
expectedSource.seek(expectedMetadata.dataStartOffset) cursorResultsCache.get(cacheKey)?.let {
actualSource.seek(actualMetadata.dataStartOffset) return@withContext it
var expectedNext = readEntryOrNull(expectedSource, expectedMetadata.totalCount > 0)
var actualNext = readEntryOrNull(actualSource, actualMetadata.totalCount > 0)
var comparedCount = 0
while (expectedNext != null && actualNext != null) {
if (expectedNext.word != actualNext.word) {
throw IndexComparisonException(
"Index words differ at entry $comparedCount: " +
"expected=${expectedNext.word}, actual=${actualNext.word}"
)
} }
try {
val word = expectedNext.word val metadata = metadataCache.getOrPut(pathOrUri) { loadMetadata(pathOrUri) }
val expectedGroup = mutableListOf<IndexEntry>() if (metadata.totalCount == 0) {
val actualGroup = mutableListOf<IndexEntry>() return@withContext CursorSearchResult(emptyList(), emptyList())
while (true) {
val current = expectedNext ?: break
if (current.word != word) break
expectedGroup.add(current)
comparedCount++
expectedNext = readEntryOrNull(
expectedSource,
comparedCount < expectedMetadata.totalCount
)
} }
val result = openSource(pathOrUri).use { source ->
var actualReadCount = comparedCount - expectedGroup.size val scans = buildList {
while (true) { add(readPrefixMatches(source, metadata, normalizedQuery))
val current = actualNext ?: break if (foldedQuery != normalizedQuery) {
if (current.word != word) break add(readPrefixMatches(source, metadata, foldedQuery))
actualGroup.add(current)
actualReadCount++
actualNext = readEntryOrNull(
actualSource,
actualReadCount < actualMetadata.totalCount
)
}
if (expectedGroup.size != actualGroup.size) {
throw IndexComparisonException(
"Headword multiplicity differs for '$word': " +
"expected=${expectedGroup.size}, actual=${actualGroup.size}"
)
}
onGroup(word, expectedGroup, actualGroup)
if (comparedCount % 10_000 == 0) yield()
}
if (expectedNext != null || actualNext != null) {
throw IndexComparisonException(
"One index ended before the other at entry $comparedCount"
)
}
IndexComparisonSummary(
expectedCount = expectedMetadata.totalCount,
actualCount = actualMetadata.totalCount,
comparedCount = comparedCount
)
} }
} }
val entries = scans.flatMap { it.entries }
.rankBySearchRelevance(SearchRankingContext(normalizedQuery, foldedQuery))
.distinctBy { it.articleKey() }
CursorSearchResult(entries, scans.mapNotNull { it.nextCursor })
}
cursorResultsCache.put(cacheKey, result)
result
} catch (e: CancellationException) {
throw e
} catch (e: Exception) {
Log.e(TAG, "searchWithCursors() - error: ${e.message}", e)
if (e is DictionaryError) throw e
throw DictionaryError.SearchError.QueryFailed(context.getString(R.string.error_search_failed, query.take(20), pathOrUri.take(20)), e)
}
} }
private fun readEntryOrNull(source: RandomAccessSource, shouldRead: Boolean): IndexEntry? = suspend fun scanTailChunk(
if (shouldRead) readEntry(source) else null pathOrUri: String,
cursor: RangeCursor,
maxEntries: Int
): List<ScannedEntry> = withContext(Dispatchers.Default) {
val metadata = metadataCache.getOrPut(pathOrUri) { loadMetadata(pathOrUri) }
if (cursor.absoluteIndex >= metadata.totalCount) return@withContext emptyList()
openSource(pathOrUri).use { source ->
source.seek(cursor.fileOffset)
val out = ArrayList<ScannedEntry>(maxEntries)
var currentIdx = cursor.absoluteIndex
var iterationCount = 0
while (currentIdx < metadata.totalCount && out.size < maxEntries) {
if (iterationCount % 1000 == 0) yield()
val entry = readEntry(source, metadata)
val entryWordLower = entry.word.lowercase()
if (!entryWordLower.startsWith(cursor.rangeQuery)) break
currentIdx++
iterationCount++
val cursorAfter = if (currentIdx < metadata.totalCount) {
RangeCursor(cursor.rangeQuery, currentIdx, source.position())
} else {
null
}
out.add(ScannedEntry(entry.copy(word = entryWordLower), cursorAfter))
}
out
}
}
fun trimMemory(level: Int) { fun trimMemory(level: Int) {
metadataCache.trim(level) metadataCache.trim(level)
resultsCache.trim(level) resultsCache.trim(level)
cursorResultsCache.trim(level)
} }
suspend fun isMetadataLoaded(pathOrUri: String): Boolean { suspend fun isMetadataLoaded(pathOrUri: String): Boolean {
@@ -252,7 +253,13 @@ class IndexSearcher(private val context: Context) {
val count = input.readInt() val count = input.readInt()
return@withContext if (version >= 2) { return@withContext if (version >= 2) {
loadMetadataV2(input, version, count) val headerBase = if (version >= 6) {
input.readInt()
12L
} else {
8L
}
loadMetadataV2(input, version, count, headerBase)
} else { } else {
loadMetadataV1(input, version, count) loadMetadataV1(input, version, count)
} }
@@ -272,13 +279,13 @@ class IndexSearcher(private val context: Context) {
// runs inside withContext(Dispatchers.IO). The dispatcher is not visible across the // runs inside withContext(Dispatchers.IO). The dispatcher is not visible across the
// suspend-call boundary, so the inspection is suppressed deliberately. // suspend-call boundary, so the inspection is suppressed deliberately.
@Suppress("BlockingMethodInNonBlockingContext") @Suppress("BlockingMethodInNonBlockingContext")
private suspend fun loadMetadataV2(input: DataInputStream, version: Int, count: Int): IndexMetadata { private suspend fun loadMetadataV2(input: DataInputStream, version: Int, count: Int, headerBase: Long): IndexMetadata {
val sparseCount = input.readInt() val sparseCount = input.readInt()
val sparseIndices = IntArray(sparseCount) val sparseIndices = IntArray(sparseCount)
val sparseOffsets = LongArray(sparseCount) val sparseOffsets = LongArray(sparseCount)
val sparseWords = Array(sparseCount) { "" } val sparseWords = Array(sparseCount) { "" }
var headerSize = 4 + 4 + 4L var headerSize = headerBase + 4L
for (i in 0 until sparseCount) { for (i in 0 until sparseCount) {
if (i % 1000 == 0) yield() if (i % 1000 == 0) yield()
@@ -337,6 +344,17 @@ class IndexSearcher(private val context: Context) {
dataStartOffset = 8L dataStartOffset = 8L
) )
} }
private suspend fun readPrefixMatches(
source: RandomAccessSource,
metadata: IndexMetadata,
query: String
): RangeScanResult {
val firstMatch = findFirstMatch(source, metadata, query)
?: return RangeScanResult(emptyList(), null)
return readMatchingEntries(source, metadata, firstMatch, query)
}
private suspend fun findFirstMatch(source: RandomAccessSource, metadata: IndexMetadata, query: String): Pair<Int, IndexEntry>? { private suspend fun findFirstMatch(source: RandomAccessSource, metadata: IndexMetadata, query: String): Pair<Int, IndexEntry>? {
val sparseIndices = metadata.sparseIndices val sparseIndices = metadata.sparseIndices
val sparseOffsets = metadata.sparseOffsets val sparseOffsets = metadata.sparseOffsets
@@ -359,7 +377,7 @@ class IndexSearcher(private val context: Context) {
while (currentAbsoluteIdx < metadata.totalCount) { while (currentAbsoluteIdx < metadata.totalCount) {
if (iterationCount % 1000 == 0) yield() if (iterationCount % 1000 == 0) yield()
val entry = readEntry(source) val entry = readEntry(source, metadata)
val entryWordLower = entry.word.lowercase() val entryWordLower = entry.word.lowercase()
if (entryWordLower >= query) { if (entryWordLower >= query) {
return if (entryWordLower.startsWith(query)) Pair(currentAbsoluteIdx, entry.copy(word = entryWordLower)) else null return if (entryWordLower.startsWith(query)) Pair(currentAbsoluteIdx, entry.copy(word = entryWordLower)) else null
@@ -394,53 +412,64 @@ class IndexSearcher(private val context: Context) {
} }
private suspend fun readMatchingEntries( private suspend fun readMatchingEntries(
source: RandomAccessSource, source: RandomAccessSource,
totalCount: Int, metadata: IndexMetadata,
firstMatch: Pair<Int, IndexEntry>, firstMatch: Pair<Int, IndexEntry>,
query: String query: String
): List<IndexEntry> { ): RangeScanResult {
val results = mutableListOf<IndexEntry>() val results = mutableListOf<IndexEntry>()
results.add(firstMatch.second) results.add(firstMatch.second)
val uniqueArticles = hashSetOf(firstMatch.second.articleKey())
var currentIdx = firstMatch.first + 1 var currentIdx = firstMatch.first + 1
var iterationCount = 0 var iterationCount = 0
while (currentIdx < totalCount && results.size < 100) { while (currentIdx < metadata.totalCount) {
if (uniqueArticles.size >= MAX_PREFIX_RESULTS) {
return RangeScanResult(results, RangeCursor(query, currentIdx, source.position()))
}
if (iterationCount % 1000 == 0) yield() if (iterationCount % 1000 == 0) yield()
val entry = readEntry(source) val entry = readEntry(source, metadata)
val entryWordLower = entry.word.lowercase() val entryWordLower = entry.word.lowercase()
if (!entryWordLower.startsWith(query)) break if (!entryWordLower.startsWith(query)) break
uniqueArticles.add(entry.articleKey())
results.add(entry.copy(word = entryWordLower)) results.add(entry.copy(word = entryWordLower))
currentIdx++ currentIdx++
iterationCount++ iterationCount++
} }
return results return RangeScanResult(results, null)
} }
private fun readEntry(source: RandomAccessSource): IndexEntry { private fun readPooledString(source: RandomAccessSource, length: Int): String {
val wordLen = source.readUnsignedShort() return if (length <= 4096) {
val word = if (wordLen <= 4096) {
val buffer = entryBuffer.get() ?: throw IllegalStateException("Entry buffer pool exhausted") val buffer = entryBuffer.get() ?: throw IllegalStateException("Entry buffer pool exhausted")
source.readFully(buffer, 0, wordLen) source.readFully(buffer, 0, length)
String(buffer, 0, wordLen, Charsets.UTF_8) String(buffer, 0, length, Charsets.UTF_8)
} else { } else {
val bytes = ByteArray(wordLen) val bytes = ByteArray(length)
source.readFully(bytes) source.readFully(bytes)
String(bytes, Charsets.UTF_8) String(bytes, Charsets.UTF_8)
} }
val origLen = source.readUnsignedShort()
val originalWord = if (origLen <= 4096) {
val buffer = entryBuffer.get() ?: throw IllegalStateException("Entry buffer pool exhausted")
source.readFully(buffer, 0, origLen)
String(buffer, 0, origLen, Charsets.UTF_8)
} else {
val bytes = ByteArray(origLen)
source.readFully(bytes)
String(bytes, Charsets.UTF_8)
} }
private fun readEntry(source: RandomAccessSource, metadata: IndexMetadata): IndexEntry {
val word = readPooledString(source, source.readUnsignedShort())
val originalWord = readPooledString(source, source.readUnsignedShort())
val offset = ArticleOffset(source.readLong()) val offset = ArticleOffset(source.readLong())
val length = ArticleLength(source.readInt()) val length = ArticleLength(source.readInt())
return IndexEntry(word, originalWord, offset, length) val isAlias = if (metadata.version >= 6) {
source.readUnsignedByte() and ENTRY_FLAG_ALIAS != 0
} else {
false
} }
return IndexEntry(word, originalWord, offset, length, isAlias = isAlias)
}
private data class ArticleKey(
val originalWord: String,
val offset: Long,
val length: Int
)
private fun IndexEntry.articleKey() = ArticleKey(originalWord, offset.value, length.value)
private fun openSource(pathOrUri: String): RandomAccessSource { private fun openSource(pathOrUri: String): RandomAccessSource {
val file = File(pathOrUri) val file = File(pathOrUri)
if (!file.exists()) throw DictionaryError.FileSystemError.NotFound(pathOrUri) if (!file.exists()) throw DictionaryError.FileSystemError.NotFound(pathOrUri)
@@ -455,20 +484,22 @@ class IndexSearcher(private val context: Context) {
private const val TAG = "IndexSearcher" private const val TAG = "IndexSearcher"
private const val SUBSTRING_SCAN_MAX_ENTRIES = 200_000 private const val SUBSTRING_SCAN_MAX_ENTRIES = 200_000
private const val SPARSE_INTERVAL_V1 = 128 private const val SPARSE_INTERVAL_V1 = 128
private const val MAX_PREFIX_RESULTS = 200
private const val MAX_SUBSTRING_RESULTS = 100
} }
} }
private class ResultsLruCache(private val maxEntries: Int) { private class ResultsLruCache<V : Any>(private val maxEntries: Int) {
private val mutex = Mutex() private val mutex = Mutex()
private val map = object : LinkedHashMap<String, List<IndexEntry>>(maxEntries, 0.75f, true) { private val map = object : LinkedHashMap<String, V>(maxEntries, 0.75f, true) {
override fun removeEldestEntry(eldest: MutableMap.MutableEntry<String, List<IndexEntry>>): Boolean { override fun removeEldestEntry(eldest: MutableMap.MutableEntry<String, V>): Boolean {
return size > maxEntries return size > maxEntries
} }
} }
suspend fun get(key: String): List<IndexEntry>? = mutex.withLock { suspend fun get(key: String): V? = mutex.withLock {
map[key] map[key]
} }
suspend fun put(key: String, value: List<IndexEntry>) = mutex.withLock { suspend fun put(key: String, value: V) = mutex.withLock {
map[key] = value map[key] = value
} }
fun trim(level: Int) { fun trim(level: Int) {
@@ -548,6 +579,8 @@ private class MetadataLruCache(private val maxBytes: Long) {
} }
interface RandomAccessSource : Closeable { interface RandomAccessSource : Closeable {
fun seek(pos: Long) fun seek(pos: Long)
fun position(): Long
fun readUnsignedByte(): Int
fun readUnsignedShort(): Int fun readUnsignedShort(): Int
fun readInt(): Int fun readInt(): Int
fun readLong(): Long fun readLong(): Long
@@ -561,6 +594,10 @@ class FileSource(file: File) : RandomAccessSource {
raf.seek(pos) raf.seek(pos)
} }
override fun position(): Long = raf.filePointer
override fun readUnsignedByte(): Int = raf.readUnsignedByte()
override fun readUnsignedShort(): Int = raf.readUnsignedShort() override fun readUnsignedShort(): Int = raf.readUnsignedShort()
override fun readInt(): Int = raf.readInt() override fun readInt(): Int = raf.readInt()
@@ -1,15 +1,17 @@
package com.example.research.data.parser package com.example.research.data.parser
import com.example.research.core.domain.model.IndexEntry import com.example.research.core.domain.model.IndexEntry
import com.example.research.core.util.foldLatinDiacritics
internal data class SearchRankingContext( internal data class SearchRankingContext(
val normalizedQuery: String, val normalizedQuery: String,
val foldedQuery: String = normalizedQuery.foldLatinDiacritics(),
) )
private data class SortKey( private data class SortKey(
val charCount: Int, val matchTier: Int,
val exactMatch: Int, val exactMatch: Int,
val startsWithMatch: Int, val charCount: Int,
val word: String, val word: String,
val entry: IndexEntry, val entry: IndexEntry,
) )
@@ -26,13 +28,18 @@ internal fun List<IndexEntry>.rankBySearchRelevance(
.ifEmpty { entry.word.trim() } .ifEmpty { entry.word.trim() }
SortKey( SortKey(
charCount = displayKey.codePointCount(0, displayKey.length), matchTier = when {
!entry.isAlias && word.startsWith(ranking.normalizedQuery) -> 0
entry.isAlias && word.startsWith(ranking.foldedQuery) -> 1
!entry.isAlias && word.foldLatinDiacritics().startsWith(ranking.foldedQuery) -> 1
else -> 2
},
exactMatch = if (word == ranking.normalizedQuery) 0 else 1, exactMatch = if (word == ranking.normalizedQuery) 0 else 1,
startsWithMatch = if (word.startsWith(ranking.normalizedQuery)) 0 else 1, charCount = displayKey.codePointCount(0, displayKey.length),
word = word, word = word,
entry = entry, entry = entry,
) )
} }
.sortedWith(compareBy({ it.charCount }, { it.exactMatch }, { it.startsWithMatch }, { it.word })) .sortedWith(compareBy({ it.matchTier }, { it.exactMatch }, { it.charCount }, { it.word }))
.map { it.entry } .map { it.entry }
} }
@@ -14,6 +14,7 @@ import com.example.research.core.util.extractDictionaryPrefix
import com.example.research.core.util.sanitizeQuery import com.example.research.core.util.sanitizeQuery
import com.example.research.core.util.toDictionaryError import com.example.research.core.util.toDictionaryError
import com.example.research.data.dictzip.DictZipRandomAccessFile import com.example.research.data.dictzip.DictZipRandomAccessFile
import com.example.research.data.local.preferences.PreferencesManager
import com.example.research.data.parser.DslCharsetDetector import com.example.research.data.parser.DslCharsetDetector
import com.example.research.data.parser.DslHeaderParser import com.example.research.data.parser.DslHeaderParser
import com.example.research.data.parser.DslIndexer import com.example.research.data.parser.DslIndexer
@@ -26,8 +27,9 @@ import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.ensureActive import kotlinx.coroutines.ensureActive
import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.flow.update import kotlinx.coroutines.flow.update
import kotlinx.coroutines.launch
import kotlinx.coroutines.supervisorScope import kotlinx.coroutines.supervisorScope
import kotlinx.coroutines.sync.Mutex import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.Semaphore import kotlinx.coroutines.sync.Semaphore
@@ -68,13 +70,9 @@ private class ProgressTracker(files: List<LocalDictionaryRepository.DiscoveredFi
aggregateWeightedMicros.addAndGet(newMicros - prevMicros) aggregateWeightedMicros.addAndGet(newMicros - prevMicros)
} }
/** fun progress(): Float {
* Current weighted sum mapped back to [0f, totalFiles] so the existing
* IndexingProgress model can keep deriving progress as aggregate/total.
*/
fun aggregateSum(): Float {
val weightedProgress = aggregateWeightedMicros.get().toDouble() / (totalWeight.toDouble() * 1_000_000.0) val weightedProgress = aggregateWeightedMicros.get().toDouble() / (totalWeight.toDouble() * 1_000_000.0)
return (weightedProgress.coerceIn(0.0, 1.0) * totalFiles).toFloat() return weightedProgress.coerceIn(0.0, 1.0).toFloat()
} }
fun reset() { fun reset() {
@@ -85,28 +83,23 @@ private class ProgressTracker(files: List<LocalDictionaryRepository.DiscoveredFi
class LocalDictionaryRepository( class LocalDictionaryRepository(
private val context: Context, private val context: Context,
private val engine: KotlinDictionaryEngine, private val engine: KotlinDictionaryEngine = KotlinDictionaryEngine(context),
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO, private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO,
private val defaultDispatcher: CoroutineDispatcher = Dispatchers.Default, private val defaultDispatcher: CoroutineDispatcher = Dispatchers.Default,
private val preferencesManager: PreferencesManager = PreferencesManager(context),
) : CoroutineScope { ) : CoroutineScope {
constructor(context: Context) : this(
context,
KotlinDictionaryEngine(context)
)
override val coroutineContext = Job() + ioDispatcher override val coroutineContext = Job() + ioDispatcher
private val mutableDictionaries = MutableStateFlow<List<Dictionary>>(emptyList()) private val mutableDictionaries = MutableStateFlow<List<Dictionary>>(emptyList())
val dictionaries: StateFlow<List<Dictionary>> val dictionaries: StateFlow<List<Dictionary>> = mutableDictionaries.asStateFlow()
get() = mutableDictionaries
private val mutableIndexingProgress = MutableStateFlow(IndexingProgress()) private val mutableIndexingProgress = MutableStateFlow(IndexingProgress())
val indexingProgress: StateFlow<IndexingProgress> val indexingProgress: StateFlow<IndexingProgress> = mutableIndexingProgress.asStateFlow()
get() = mutableIndexingProgress
private val indexingMutex = Mutex() private val indexingMutex = Mutex()
private val indexingSemaphore = Semaphore(1) private val indexingSemaphore = Semaphore(1)
private val warmupMutex = Mutex() private val warmupMutex = Mutex()
private val dictionaryStateMutex = Mutex()
@Volatile @Volatile
private var currentIndexingJob: Job? = null private var currentIndexingJob: Job? = null
@@ -124,6 +117,24 @@ class LocalDictionaryRepository(
fun isIndexingInProgress(): Boolean = currentIndexingJob?.isActive == true fun isIndexingInProgress(): Boolean = currentIndexingJob?.isActive == true
suspend fun listDictionaryPayloadFileNames(path: String): List<String>? =
withContext(ioDispatcher) {
try {
val directory = File(path)
when {
!directory.exists() -> {
if (directory.parentFile?.isDirectory == true) emptyList() else null
}
!directory.isDirectory -> null
else -> directory.listFiles { file -> isDictionaryPayloadFile(file) }
?.map(File::getName)
}
} catch (e: SecurityException) {
Log.w(TAG, "Failed to list dictionary files: ${e.message}")
null
}
}
suspend fun scanDirectory(pathOrUri: String): OperationResult<Int> = withContext(ioDispatcher) { suspend fun scanDirectory(pathOrUri: String): OperationResult<Int> = withContext(ioDispatcher) {
if (isIndexingInProgress()) { if (isIndexingInProgress()) {
return@withContext OperationResult.Error( return@withContext OperationResult.Error(
@@ -140,7 +151,7 @@ class LocalDictionaryRepository(
val filesToScan = scanLocalDirectory(pathOrUri) val filesToScan = scanLocalDirectory(pathOrUri)
if (filesToScan.isEmpty()) { if (filesToScan.isEmpty()) {
mutableDictionaries.value = emptyList() replaceDictionaries(emptyList())
mutableIndexingProgress.value = IndexingProgress(isIndexing = false) mutableIndexingProgress.value = IndexingProgress(isIndexing = false)
return@withContext OperationResult.Success(0) return@withContext OperationResult.Success(0)
} }
@@ -159,12 +170,12 @@ class LocalDictionaryRepository(
} }
if (filesToIndex.isEmpty()) { if (filesToIndex.isEmpty()) {
mutableDictionaries.value = existingDictionaries.sortedBy { it.name } replaceDictionaries(existingDictionaries)
engine.getIndexSearcher().trimMemory(80) engine.getIndexSearcher().trimMemory(80)
return@withContext OperationResult.Success(existingDictionaries.size) return@withContext OperationResult.Success(existingDictionaries.size)
} }
mutableDictionaries.value = existingDictionaries.sortedBy { it.name } replaceDictionaries(existingDictionaries)
val totalArticlesIndexed = AtomicInteger(existingDictionaries.sumOf { it.articleCount }) val totalArticlesIndexed = AtomicInteger(existingDictionaries.sumOf { it.articleCount })
@@ -177,68 +188,63 @@ class LocalDictionaryRepository(
mutableIndexingProgress.emit( mutableIndexingProgress.emit(
IndexingProgress( IndexingProgress(
isIndexing = true, isIndexing = true,
currentFile = context.getString(R.string.indexing_label), progress = progressTracker?.progress() ?: 0f,
totalFiles = filesToScan.size,
currentIndex = filesToScan.size - filesToIndex.size,
aggregateSum = progressTracker?.aggregateSum() ?: -1f,
) )
) )
val lastEmittedPercent = AtomicInteger(-1) val lastEmittedPercent = AtomicInteger(-1)
val lastProgressEmitMs = AtomicLong(0L) val lastProgressEmitMs = AtomicLong(0L)
val publishCurrentProgress = {
val tracker = progressTracker
if (tracker != null) {
val progress = tracker.progress()
lastEmittedPercent.set((progress * 100f).toInt().coerceIn(0, 100))
lastProgressEmitMs.set(SystemClock.elapsedRealtime())
mutableIndexingProgress.update { p ->
p.copy(progress = progress)
}
}
}
supervisorScope { supervisorScope {
filesToIndex.mapIndexed { index, file -> filesToIndex.map { file ->
async { async {
indexingSemaphore.withPermit { indexingSemaphore.withPermit {
coroutineContext.ensureActive() coroutineContext.ensureActive()
yield() yield()
val res = indexDictionary(file) { _, op, prog -> val res = indexDictionary(file) { _, _, prog ->
if (prog >= 0f) { if (prog >= 0f) {
progressTracker?.updateFileProgress(file.name, prog) progressTracker?.updateFileProgress(file.name, prog)
} }
val tracker = progressTracker val progress = progressTracker?.progress() ?: 0f
val aggregate = tracker?.aggregateSum() ?: -1f val newPercent = (progress * 100f).toInt().coerceIn(0, 100)
val newPercent = if (filesToScan.isNotEmpty() && aggregate >= 0f) {
((aggregate / filesToScan.size) * 100f).toInt().coerceIn(0, 100)
} else -1
val currentIdx = filesToScan.size - filesToIndex.size + index + 1
val previous = mutableIndexingProgress.value
val now = SystemClock.elapsedRealtime() val now = SystemClock.elapsedRealtime()
val percentChanged = newPercent >= 0 && newPercent != lastEmittedPercent.get() val percentChanged = newPercent != lastEmittedPercent.get()
val fileChanged = previous.currentFile != file.name
val completed = newPercent >= 100 || prog >= 1f val completed = newPercent >= 100 || prog >= 1f
val intervalElapsed = val intervalElapsed =
now - lastProgressEmitMs.get() >= MIN_PROGRESS_EMIT_INTERVAL_MS now - lastProgressEmitMs.get() >= MIN_PROGRESS_EMIT_INTERVAL_MS
val shouldEmit = fileChanged || completed || (percentChanged && intervalElapsed) val shouldEmit = completed || (percentChanged && intervalElapsed)
if (shouldEmit) { if (shouldEmit) {
if (newPercent >= 0) lastEmittedPercent.set(newPercent) lastEmittedPercent.set(newPercent)
lastProgressEmitMs.set(now) lastProgressEmitMs.set(now)
mutableIndexingProgress.update { p -> mutableIndexingProgress.update { p ->
p.copy( p.copy(progress = progress)
currentFile = file.name,
currentIndex = currentIdx,
currentFileProgress = prog,
label = op.ifEmpty { p.label },
aggregateSum = aggregate,
)
} }
} }
} }
if (res is OperationResult.Success) { if (res is OperationResult.Success) {
progressTracker?.updateFileProgress(file.name, 1.0f) progressTracker?.updateFileProgress(file.name, 1.0f)
publishCurrentProgress()
totalArticlesIndexed.addAndGet(res.data.articleCount) totalArticlesIndexed.addAndGet(res.data.articleCount)
mutableDictionaries.update { current -> val dictionary = res.data
(current + res.data) addDictionary(dictionary)
.distinctBy { it.path } dictionary
.sortedBy { it.name }
}
res.data
} else null } else null
} }
} }
@@ -266,17 +272,37 @@ class LocalDictionaryRepository(
} }
} }
private suspend fun replaceDictionaries(dictionaries: List<Dictionary>) {
dictionaryStateMutex.withLock { applyDictionaries(dictionaries) }
}
private suspend fun addDictionary(dictionary: Dictionary) {
dictionaryStateMutex.withLock {
applyDictionaries((mutableDictionaries.value + dictionary).distinctBy { it.path })
}
}
private suspend fun applyDictionaries(dictionaries: List<Dictionary>) {
val disabledPaths = preferencesManager.disabledDictionaryPaths.first()
mutableDictionaries.value = dictionaries
.map { it.withActiveStatus(it.path !in disabledPaths) }
.sortedBy { it.name }
}
private suspend fun tryLoadExistingIndex(file: DiscoveredFile): Dictionary? { private suspend fun tryLoadExistingIndex(file: DiscoveredFile): Dictionary? {
val dictFile = File(file.localPath) val dictFile = File(file.localPath)
val indexFile = File(file.indexPath) val indexFile = File(file.indexPath)
if (!indexFile.exists() || indexFile.length() == 0L) return null if (!indexFile.exists() || indexFile.length() == 0L) {
deleteIndexFiles(indexFile)
return null
}
if (!dictFile.exists()) return null if (!dictFile.exists()) return null
val header = readIndexHeader(indexFile) val header = readIndexHeader(indexFile)
if (header == null || header.version != DslIndexer.INDEX_VERSION) { if (header == null || header.version != DslIndexer.INDEX_VERSION) {
try { indexFile.delete() } catch (_: Exception) {} deleteIndexFiles(indexFile)
return null return null
} }
@@ -326,23 +352,36 @@ class LocalDictionaryRepository(
) )
OperationResult.Success(dict) OperationResult.Success(dict)
} else { } else {
indexFile.delete() deleteIndexFiles(indexFile)
OperationResult.Error(result.errorMsg, null) OperationResult.Error(result.errorMsg, null)
} }
} catch(e: Exception) { } catch(e: Exception) {
indexFile.delete() deleteIndexFiles(indexFile)
if (e is CancellationException) throw e if (e is CancellationException) throw e
OperationResult.Error(e.message ?: context.getString(R.string.error_indexing), e) OperationResult.Error(e.message ?: context.getString(R.string.error_indexing), e)
} }
} }
private fun deleteIndexFiles(indexFile: File, errors: MutableList<String>? = null) {
deleteFileTracked(indexFile, "index file", errors)
deleteFileTracked(DslIndexer.foldedIndexFile(indexFile), "legacy folded index file", errors)
}
private fun deleteFileTracked(file: File, label: String, errors: MutableList<String>?) {
try {
if (file.exists() && !file.delete()) {
errors?.add("Failed to delete $label: ${file.absolutePath}")
}
} catch (e: Exception) {
errors?.add("Failed to delete $label: ${file.absolutePath} (${e.message})")
}
}
private fun scanLocalDirectory(path: String): List<DiscoveredFile> { private fun scanLocalDirectory(path: String): List<DiscoveredFile> {
val dir = File(path) val dir = File(path)
if (!dir.exists() || !dir.isDirectory) return emptyList() if (!dir.exists() || !dir.isDirectory) return emptyList()
return dir.listFiles { f -> return dir.listFiles { file -> isDictionaryPayloadFile(file) }?.map { file ->
f.isFile && (f.name.endsWith(".dsl") || f.name.endsWith(".dsl.dz") || f.name.endsWith(".dsl.gz"))
}?.map { file ->
DiscoveredFile( DiscoveredFile(
name = file.name, name = file.name,
localPath = file.absolutePath, localPath = file.absolutePath,
@@ -351,10 +390,17 @@ class LocalDictionaryRepository(
} ?: emptyList() } ?: emptyList()
} }
private fun isDictionaryPayloadFile(file: File): Boolean =
file.isFile && (
file.name.endsWith(".dsl") ||
file.name.endsWith(".dsl.dz") ||
file.name.endsWith(".dsl.gz")
)
suspend fun search(query: String): OperationResult<List<IndexEntry>> = withContext(defaultDispatcher) { suspend fun search(query: String): OperationResult<List<IndexEntry>> = withContext(defaultDispatcher) {
if (query.isBlank()) return@withContext OperationResult.Success(emptyList()) if (query.isBlank()) return@withContext OperationResult.Success(emptyList())
val activeDicts = mutableDictionaries.value val activeDicts = mutableDictionaries.value.filter { it.isActive }
val sanitizedQuery = query.sanitizeQuery() val sanitizedQuery = query.sanitizeQuery()
ReSearchTrace.asyncSection(ReSearchTrace.SEARCH_DIRECT) { ReSearchTrace.asyncSection(ReSearchTrace.SEARCH_DIRECT) {
@@ -404,45 +450,63 @@ class LocalDictionaryRepository(
return engine.getIndexSearcher() return engine.getIndexSearcher()
} }
fun toggleDictionaryActive(dictionaryPath: String) { suspend fun toggleDictionaryActive(dictionaryPath: String) {
dictionaryStateMutex.withLock {
val isActive = mutableDictionaries.value
.firstOrNull { it.path == dictionaryPath }
?.isActive
?.not()
?: return
preferencesManager.setDictionaryActive(dictionaryPath, isActive)
mutableDictionaries.update { currentList -> mutableDictionaries.update { currentList ->
currentList.map { dict -> currentList.map { dict ->
if (dict.path == dictionaryPath) { if (dict.path == dictionaryPath) {
dict.withActiveStatus(!dict.isActive) dict.withActiveStatus(isActive)
} else { } else {
dict dict
} }
} }
} }
} }
fun deleteDictionary(dictionary: Dictionary): OperationResult<Unit> {
try {
mutableDictionaries.update { current ->
current.filter { it.path != dictionary.path }
} }
launch(ioDispatcher) { private fun removeDictionaryFromState(path: String) {
try { mutableDictionaries.update { current -> current.filter { it.path != path } }
val deleteErrors = mutableListOf<String>() }
suspend fun deleteDictionary(dictionary: Dictionary): OperationResult<Unit> {
return try {
val (failure, cleanupErrors) = withContext(ioDispatcher) {
val dictFile = File(dictionary.path) val dictFile = File(dictionary.path)
if (dictFile.exists()) { if (dictFile.exists() && !dictFile.delete()) {
if (!dictFile.delete()) { return@withContext Pair(
deleteErrors.add("Failed to delete dictionary file: ${dictFile.absolutePath}") "Failed to delete dictionary file: ${dictFile.absolutePath}",
} emptyList(),
)
} }
val deleteErrors = mutableListOf<String>()
val indexFile = if (dictionary.indexPath.endsWith(".idx", ignoreCase = true)) { val indexFile = if (dictionary.indexPath.endsWith(".idx", ignoreCase = true)) {
File(dictionary.indexPath) File(dictionary.indexPath)
} else { } else {
File("${dictionary.indexPath}.idx") File("${dictionary.indexPath}.idx")
} }
if (indexFile.exists()) { deleteIndexFiles(indexFile, deleteErrors)
if (!indexFile.delete()) { Pair<String?, List<String>>(null, deleteErrors)
deleteErrors.add("Failed to delete index file: ${indexFile.absolutePath}")
} }
if (failure != null) {
return OperationResult.Error(failure)
}
dictionaryStateMutex.withLock {
try {
preferencesManager.removeDictionaryActiveState(dictionary.path)
} catch (e: Exception) {
Log.w(TAG, "Failed to clear dictionary active state: ${e.message}")
}
removeDictionaryFromState(dictionary.path)
} }
try { try {
@@ -451,18 +515,16 @@ class LocalDictionaryRepository(
Log.w(TAG, "Failed to trim memory after deletion: ${e.message}") Log.w(TAG, "Failed to trim memory after deletion: ${e.message}")
} }
if (deleteErrors.isNotEmpty()) { if (cleanupErrors.isNotEmpty()) {
val message = deleteErrors.joinToString("; ") Log.w(
Log.w(TAG, "Dictionary deletion completed with errors: $message") TAG,
} "Dictionary deletion completed with errors: ${cleanupErrors.joinToString("; ")}"
} catch (e: Exception) { )
Log.e(TAG, "Background deletion failed: ${e.message}", e)
}
} }
return OperationResult.Success(Unit) OperationResult.Success(Unit)
} catch (e: Exception) { } catch (e: Exception) {
return OperationResult.Error("Failed to delete dictionary", e) OperationResult.Error("Failed to delete dictionary", e)
} }
} }
@@ -507,13 +569,22 @@ class LocalDictionaryRepository(
if (count < 0) return null if (count < 0) return null
var articleCount = count
if (version >= 6) {
if (indexFile.length() < INDEX_V6_HEADER_BYTES) return null
articleCount = dataInput.readInt()
if (articleCount < 0) return null
}
if (version >= 2) { if (version >= 2) {
if (indexFile.length() < INDEX_V2_HEADER_BYTES) return null val fixedHeaderBytes =
if (version >= 6) INDEX_V6_HEADER_BYTES else INDEX_V2_HEADER_BYTES
if (indexFile.length() < fixedHeaderBytes) return null
val sparseCount = dataInput.readInt() val sparseCount = dataInput.readInt()
if (sparseCount < 0) return null if (sparseCount < 0) return null
var headerBytes = INDEX_V2_HEADER_BYTES var headerBytes = fixedHeaderBytes
repeat(sparseCount) { repeat(sparseCount) {
dataInput.readInt() dataInput.readInt()
dataInput.readLong() dataInput.readLong()
@@ -524,7 +595,7 @@ class LocalDictionaryRepository(
} }
} }
IndexHeader(version, count) IndexHeader(version, articleCount)
} }
} }
} catch (e: Exception) { } catch (e: Exception) {
@@ -602,6 +673,7 @@ class LocalDictionaryRepository(
const val MIN_PROGRESS_EMIT_INTERVAL_MS = 200L const val MIN_PROGRESS_EMIT_INTERVAL_MS = 200L
const val LEGACY_INDEX_HEADER_BYTES = 8L const val LEGACY_INDEX_HEADER_BYTES = 8L
const val INDEX_V2_HEADER_BYTES = 12 const val INDEX_V2_HEADER_BYTES = 12
const val INDEX_V6_HEADER_BYTES = 16
const val SPARSE_ENTRY_FIXED_BYTES = 14 const val SPARSE_ENTRY_FIXED_BYTES = 14
} }
} }
@@ -12,6 +12,7 @@ import kotlin.time.Duration.Companion.seconds
class DownloadManager( class DownloadManager(
private val dictionaryRepository: DictionaryRepository, private val dictionaryRepository: DictionaryRepository,
private val unknownErrorMessage: String, private val unknownErrorMessage: String,
private val sourceUnavailableMessage: String,
dispatcher: CoroutineDispatcher = Dispatchers.IO, dispatcher: CoroutineDispatcher = Dispatchers.IO,
externalScope: CoroutineScope? = null externalScope: CoroutineScope? = null
) { ) {
@@ -24,10 +25,11 @@ class DownloadManager(
mutableDownloadProgressState.asStateFlow() mutableDownloadProgressState.asStateFlow()
private val mutex = Mutex() private val mutex = Mutex()
private var downloadJob: Job? = null private var downloadJob: Job? = null
private var cancelCleanupJob: Job? = null
companion object { companion object {
private const val TAG = "DownloadManager" private const val TAG = "DownloadManager"
private const val DOWNLOAD_PROGRESS_STEP_PERCENT = 3 private const val DOWNLOAD_PROGRESS_STEP_PERCENT = 1
private const val EXTRACT_PROGRESS_STEP_PERCENT = 5 private const val EXTRACT_PROGRESS_STEP_PERCENT = 5
private val TERMINAL_STATE_DURATION = 2.seconds private val TERMINAL_STATE_DURATION = 2.seconds
} }
@@ -39,30 +41,51 @@ class DownloadManager(
val clampedProgress = progress.coerceIn(0f, 1f) val clampedProgress = progress.coerceIn(0f, 1f)
mutableDownloadProgressState.value = DownloadProgressState(state, clampedProgress) mutableDownloadProgressState.value = DownloadProgressState(state, clampedProgress)
mutableDownloadState.value = state mutableDownloadState.value = state
if (state is DownloadState.Success) {
onDownloadSuccess?.invoke()
} else if (state is DownloadState.Error || state is DownloadState.Cancelled) {
onTerminal?.invoke(state)
} }
}
var onFlowStarted: (() -> Unit)? = null
var onDownloadSuccess: (() -> Unit)? = null
var onTerminal: ((DownloadState) -> Unit)? = null
fun startDownload() { fun startDownload() {
if (downloadState.value is DownloadState.Loading) return if (downloadState.value is DownloadState.Loading) return
if (!downloadScope.isActive) return if (!downloadScope.isActive) return
onFlowStarted?.invoke()
cancelCleanupJob = null
downloadJob = downloadScope.launch { downloadJob = downloadScope.launch {
mutex.withLock { mutex.withLock {
try { try {
updateDownloadProgressState(DownloadState.Loading, 0f) val enabledSources = try {
dictionaryRepository.getEnabledSources()
val hasEnabledSources = try {
dictionaryRepository.hasEnabledSources()
} catch (_: Exception) { } catch (_: Exception) {
false emptyList()
} }
if (!hasEnabledSources) { if (enabledSources.isEmpty()) {
dictionaryRepository.performAllCleanup() dictionaryRepository.performAllCleanup()
updateDownloadProgressState(DownloadState.Success, 1f) updateDownloadProgressState(DownloadState.Idle, 0f)
return@withLock return@withLock
} }
val downloadableSources =
dictionaryRepository.filterServerAvailableSources(enabledSources)
if (downloadableSources.isEmpty()) {
updateDownloadProgressState(
DownloadState.Error(sourceUnavailableMessage), 0f
)
return@withLock
}
updateDownloadProgressState(DownloadState.Loading, 0f)
var lastReportedProgressBucket = -1 var lastReportedProgressBucket = -1
val result = ReSearchTrace.asyncSection(ReSearchTrace.DICTIONARY_DOWNLOAD) { val result = ReSearchTrace.asyncSection(ReSearchTrace.DICTIONARY_DOWNLOAD) {
dictionaryRepository.downloadDictionaries { progress -> dictionaryRepository.downloadSources(downloadableSources) { progress ->
val progressBucket = progressBucket(progress, DOWNLOAD_PROGRESS_STEP_PERCENT) val progressBucket = progressBucket(progress, DOWNLOAD_PROGRESS_STEP_PERCENT)
if (progressBucket != lastReportedProgressBucket) { if (progressBucket != lastReportedProgressBucket) {
lastReportedProgressBucket = progressBucket lastReportedProgressBucket = progressBucket
@@ -94,7 +117,7 @@ class DownloadManager(
} catch (e: Exception) { } catch (e: Exception) {
if (e is CancellationException) { if (e is CancellationException) {
updateDownloadProgressState(DownloadState.Cancelled, 0f) updateDownloadProgressState(DownloadState.Cancelled, 0f)
downloadScope.launch { cancelCleanupJob = downloadScope.launch {
try { try {
dictionaryRepository.deleteUnprocessedFiles() dictionaryRepository.deleteUnprocessedFiles()
} catch (cleanupException: Exception) { } catch (cleanupException: Exception) {
@@ -124,19 +147,41 @@ class DownloadManager(
fun startDownloadForSources(sourceUrls: List<String>) { fun startDownloadForSources(sourceUrls: List<String>) {
if (downloadState.value is DownloadState.Loading) return if (downloadState.value is DownloadState.Loading) return
if (!downloadScope.isActive) return if (!downloadScope.isActive) return
onFlowStarted?.invoke()
cancelCleanupJob = null
downloadJob = downloadScope.launch { downloadJob = downloadScope.launch {
mutex.withLock { mutex.withLock {
try { try {
updateDownloadProgressState(DownloadState.Loading, 0f)
if (sourceUrls.isEmpty()) { if (sourceUrls.isEmpty()) {
updateDownloadProgressState(DownloadState.Success, 1f) updateDownloadProgressState(DownloadState.Idle, 0f)
return@withLock return@withLock
} }
val sources = try {
dictionaryRepository.getEnabledSources(sourceUrls)
} catch (_: Exception) {
emptyList()
}
if (sources.isEmpty()) {
updateDownloadProgressState(DownloadState.Idle, 0f)
return@withLock
}
val downloadableSources =
dictionaryRepository.filterServerAvailableSources(sources)
if (downloadableSources.isEmpty()) {
updateDownloadProgressState(
DownloadState.Error(sourceUnavailableMessage), 0f
)
return@withLock
}
updateDownloadProgressState(DownloadState.Loading, 0f)
var lastReportedProgressBucket = -1 var lastReportedProgressBucket = -1
val result = ReSearchTrace.asyncSection(ReSearchTrace.DICTIONARY_DOWNLOAD) { val result = ReSearchTrace.asyncSection(ReSearchTrace.DICTIONARY_DOWNLOAD) {
dictionaryRepository.downloadSpecificSources(sourceUrls) { progress -> dictionaryRepository.downloadSources(downloadableSources) { progress ->
val progressBucket = progressBucket(progress, DOWNLOAD_PROGRESS_STEP_PERCENT) val progressBucket = progressBucket(progress, DOWNLOAD_PROGRESS_STEP_PERCENT)
if (progressBucket != lastReportedProgressBucket) { if (progressBucket != lastReportedProgressBucket) {
lastReportedProgressBucket = progressBucket lastReportedProgressBucket = progressBucket
@@ -168,7 +213,7 @@ class DownloadManager(
} catch (e: Exception) { } catch (e: Exception) {
if (e is CancellationException) { if (e is CancellationException) {
updateDownloadProgressState(DownloadState.Cancelled, 0f) updateDownloadProgressState(DownloadState.Cancelled, 0f)
downloadScope.launch { cancelCleanupJob = downloadScope.launch {
try { try {
dictionaryRepository.deleteUnprocessedFiles() dictionaryRepository.deleteUnprocessedFiles()
} catch (cleanupException: Exception) { } catch (cleanupException: Exception) {
@@ -195,10 +240,12 @@ class DownloadManager(
} }
} }
fun cancelDownload() { fun cancelDownload() {
downloadJob?.cancel() val jobToCancel = downloadJob
jobToCancel?.cancel()
updateDownloadProgressState(DownloadState.Cancelled, 0f) updateDownloadProgressState(DownloadState.Cancelled, 0f)
downloadScope.launch { downloadScope.launch {
delay(TERMINAL_STATE_DURATION) jobToCancel?.join()
cancelCleanupJob?.join()
if (downloadState.value == DownloadState.Cancelled) { if (downloadState.value == DownloadState.Cancelled) {
updateDownloadProgressState(DownloadState.Idle, 0f) updateDownloadProgressState(DownloadState.Idle, 0f)
} }
@@ -1,4 +1,8 @@
package com.example.research.feature.download.model package com.example.research.feature.download.model
import androidx.compose.runtime.Immutable
@Immutable
sealed class DownloadState { sealed class DownloadState {
data object Idle : DownloadState() data object Idle : DownloadState()
data object Loading : DownloadState() data object Loading : DownloadState()
@@ -7,3 +11,6 @@ sealed class DownloadState {
data class Error(val message: String) : DownloadState() data class Error(val message: String) : DownloadState()
data object Cancelled : DownloadState() data object Cancelled : DownloadState()
} }
val DownloadState.isActive: Boolean
get() = this is DownloadState.Loading || this is DownloadState.Extracting || this is DownloadState.Success
@@ -12,6 +12,12 @@ import kotlinx.coroutines.withContext
import okhttp3.OkHttpClient import okhttp3.OkHttpClient
import okhttp3.Request import okhttp3.Request
enum class SourceAvailability {
AVAILABLE,
NOT_FOUND,
UNKNOWN
}
class DictionaryChecker( class DictionaryChecker(
private val fileStorageManager: FileStorageManager, private val fileStorageManager: FileStorageManager,
private val client: OkHttpClient private val client: OkHttpClient
@@ -85,7 +91,26 @@ class DictionaryChecker(
} }
} }
private fun isSourceAvailableOnServer(source: DictionarySource, date: String): Boolean { suspend fun checkSourceAvailability(source: DictionarySource): SourceAvailability =
withContext(Dispatchers.IO) {
val currentDate = DateUtils.getCurrentDateString()
when (probeSource(source, currentDate)) {
SourceAvailability.AVAILABLE -> SourceAvailability.AVAILABLE
SourceAvailability.UNKNOWN -> SourceAvailability.UNKNOWN
SourceAvailability.NOT_FOUND -> {
if (DictionarySource.hasDatePlaceholder(source.urlTemplate)) {
probeSource(source, DateUtils.getPreviousDateString())
} else {
SourceAvailability.NOT_FOUND
}
}
}
}
private fun isSourceAvailableOnServer(source: DictionarySource, date: String): Boolean =
probeSource(source, date) == SourceAvailability.AVAILABLE
private fun probeSource(source: DictionarySource, date: String): SourceAvailability {
return try { return try {
val url = DictionarySource.buildUrl(source.urlTemplate, date) val url = DictionarySource.buildUrl(source.urlTemplate, date)
val request = Request.Builder() val request = Request.Builder()
@@ -94,12 +119,17 @@ class DictionaryChecker(
.build() .build()
client.newCall(request).execute().use { response -> client.newCall(request).execute().use { response ->
response.isSuccessful && if (response.isSuccessful &&
response.header("Content-Type")?.startsWith("text/html") != true response.header("Content-Type")?.startsWith("text/html") != true
) {
SourceAvailability.AVAILABLE
} else {
SourceAvailability.NOT_FOUND
}
} }
} catch (e: Exception) { } catch (e: Exception) {
Log.w(TAG, "Failed to check source availability for ${source.urlTemplate}: ${e.message}") Log.w(TAG, "Failed to check source availability for ${source.urlTemplate}: ${e.message}")
false SourceAvailability.UNKNOWN
} }
} }
@@ -59,11 +59,11 @@ class DictionaryCleaner(
toDelete.add(item.name) toDelete.add(item.name)
continue continue
} }
if ((item.ext == "idx" || item.ext == "dsl.idx") && item.date != latestDsl) { if (item.ext in indexExtensions && item.date != latestDsl) {
toDelete.add(item.name) toDelete.add(item.name)
continue continue
} }
if ((item.ext == "idx" || item.ext == "dsl.idx") && item.date !in dslDates) { if (item.ext in indexExtensions && item.date !in dslDates) {
toDelete.add(item.name) toDelete.add(item.name)
} }
} }
@@ -76,10 +76,13 @@ class DictionaryCleaner(
val date: String, val date: String,
val ext: String, val ext: String,
) )
private val validExtensions = setOf("dsl", "dsl.dz", "dsl.gz", "gz", "idx", "dsl.idx") private val indexExtensions = setOf("idx", "dsl.idx", "idx.fold", "dsl.idx.fold")
private val validExtensions = setOf("dsl", "dsl.dz", "dsl.gz", "gz") + indexExtensions
private fun parseManagedName(name: String, prefixes: Set<String>): ManagedFileName? { private fun parseManagedName(name: String, prefixes: Set<String>): ManagedFileName? {
val ext = when { val ext = when {
name.endsWith(".dsl.idx.fold") -> "dsl.idx.fold"
name.endsWith(".idx.fold") -> "idx.fold"
name.endsWith(".dsl.dz") -> "dsl.dz" name.endsWith(".dsl.dz") -> "dsl.dz"
name.endsWith(".dsl.gz") -> "dsl.gz" name.endsWith(".dsl.gz") -> "dsl.gz"
name.endsWith(".dsl.idx") -> "dsl.idx" name.endsWith(".dsl.idx") -> "dsl.idx"
@@ -343,6 +343,8 @@ class DictionaryDownloader(
context.getString(R.string.download_connection_reset) context.getString(R.string.download_connection_reset)
e is SecurityException -> e is SecurityException ->
context.getString(R.string.download_no_write_permission) context.getString(R.string.download_no_write_permission)
e.isNetworkError() ->
context.getString(R.string.download_network_failed)
else -> else ->
context.getString(R.string.download_error) context.getString(R.string.download_error)
} }
@@ -8,6 +8,8 @@ import com.example.research.core.domain.model.DictionarySource
import com.example.research.data.local.preferences.PreferencesManager import com.example.research.data.local.preferences.PreferencesManager
import com.example.research.common.util.FileStorageManager import com.example.research.common.util.FileStorageManager
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.flow.first import kotlinx.coroutines.flow.first
import kotlinx.coroutines.withContext import kotlinx.coroutines.withContext
import okhttp3.OkHttpClient import okhttp3.OkHttpClient
@@ -40,74 +42,58 @@ class DictionaryRepository(
checker.areDictionariesUpToDate(uri, sourcesToCheck, files) checker.areDictionariesUpToDate(uri, sourcesToCheck, files)
} }
suspend fun hasEnabledSources(): Boolean = withContext(Dispatchers.IO) { suspend fun getEnabledSources(sourceUrls: List<String>? = null): List<DictionarySource> =
preferencesManager.dictionarySources.first().any { it.isEnabled } withContext(Dispatchers.IO) {
val enabledSources = preferencesManager.dictionarySources.first()
.filter { it.isEnabled }
if (sourceUrls == null) {
return@withContext enabledSources
} }
suspend fun downloadDictionaries(
onProgress: (Float) -> Unit
): Result<Unit> = withContext(Dispatchers.IO) {
val folderUri = preferencesManager.dictionaryPath
val uri = Uri.fromFile(File(folderUri))
val files = fileStorageManager.listFilesInFolder(uri)
cleaner.deleteCorruptedArchives(uri, files)
synchronized(filesBeforeDownload) {
filesBeforeDownload = files.map { it.name }.toMutableSet()
}
val sources = preferencesManager.dictionarySources.first()
val enabledSources = sources.filter { it.isEnabled }
synchronized(downloadedPrefixes) {
downloadedPrefixes = enabledSources.mapNotNull {
DictionarySource.extractPrefix(it.urlTemplate)
}.toMutableSet()
}
downloader.downloadDictionaries(
folderUri = uri,
sources = enabledSources,
onProgress = onProgress
)
}
suspend fun downloadSpecificSources(
sourceUrls: List<String>,
onProgress: (Float) -> Unit
): Result<Unit> = withContext(Dispatchers.IO) {
val folderUri = preferencesManager.dictionaryPath
val uri = Uri.fromFile(File(folderUri))
val files = fileStorageManager.listFilesInFolder(uri)
cleaner.deleteCorruptedArchives(uri, files)
synchronized(filesBeforeDownload) {
filesBeforeDownload = files.map { it.name }.toMutableSet()
}
val allSources = preferencesManager.dictionarySources.first()
val sourceKeys = sourceUrls.mapTo(mutableSetOf(), DictionarySource::identityKey) val sourceKeys = sourceUrls.mapTo(mutableSetOf(), DictionarySource::identityKey)
val sourcesToDownload = allSources.mapNotNull { source -> enabledSources.mapNotNull { source ->
if (!source.isEnabled || DictionarySource.identityKey(source.urlTemplate) !in sourceKeys) { if (DictionarySource.identityKey(source.urlTemplate) !in sourceKeys) {
null null
} else { } else {
source.copy(urlTemplate = DictionarySource.normalizeTemplate(source.urlTemplate)) source.copy(urlTemplate = DictionarySource.normalizeTemplate(source.urlTemplate))
} }
} }
}
if (sourcesToDownload.isEmpty()) { suspend fun filterServerAvailableSources(
return@withContext Result.success(Unit) sources: List<DictionarySource>
): List<DictionarySource> = withContext(Dispatchers.IO) {
if (sources.isEmpty()) {
return@withContext emptyList()
}
sources
.map { source -> async { source to checker.checkSourceAvailability(source) } }
.awaitAll()
.filter { (_, availability) -> availability != SourceAvailability.NOT_FOUND }
.map { (source, _) -> source }
}
suspend fun downloadSources(
sources: List<DictionarySource>,
onProgress: (Float) -> Unit
): Result<Unit> = withContext(Dispatchers.IO) {
val folderUri = preferencesManager.dictionaryPath
val uri = Uri.fromFile(File(folderUri))
val files = fileStorageManager.listFilesInFolder(uri)
cleaner.deleteCorruptedArchives(uri, files)
synchronized(filesBeforeDownload) {
filesBeforeDownload = files.map { it.name }.toMutableSet()
} }
synchronized(downloadedPrefixes) { synchronized(downloadedPrefixes) {
downloadedPrefixes = sourcesToDownload.mapNotNull { downloadedPrefixes = sources.mapNotNull {
DictionarySource.extractPrefix(it.urlTemplate) DictionarySource.extractPrefix(it.urlTemplate)
}.toMutableSet() }.toMutableSet()
} }
downloader.downloadDictionaries( downloader.downloadDictionaries(
folderUri = uri, folderUri = uri,
sources = sourcesToDownload, sources = sources,
onProgress = onProgress onProgress = onProgress
) )
} }
@@ -3,19 +3,19 @@ package com.example.research.feature.download.service
import android.app.Service import android.app.Service
import android.content.Intent import android.content.Intent
import android.os.IBinder import android.os.IBinder
import android.util.Log
import androidx.core.app.ServiceCompat import androidx.core.app.ServiceCompat
import com.example.research.ReSearchApplication import com.example.research.ReSearchApplication
import com.example.research.common.progress.renderTitle import com.example.research.common.progress.DictionaryPipelineCoordinator
import com.example.research.common.progress.DictionaryProgressPresenter
import com.example.research.common.util.NotificationHelper import com.example.research.common.util.NotificationHelper
import com.example.research.core.util.OperationResult
import com.example.research.data.repository.LocalDictionaryRepository import com.example.research.data.repository.LocalDictionaryRepository
import com.example.research.feature.download.DownloadManager import com.example.research.feature.download.DownloadManager
import com.example.research.feature.download.model.DownloadState import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.* import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.flow.* import kotlinx.coroutines.SupervisorJob
import kotlin.time.Duration.Companion.milliseconds import kotlinx.coroutines.cancel
import kotlin.time.Duration.Companion.seconds import kotlinx.coroutines.cancelChildren
import kotlinx.coroutines.Job
class DictionaryForegroundService : Service() { class DictionaryForegroundService : Service() {
@@ -23,21 +23,21 @@ class DictionaryForegroundService : Service() {
const val ACTION_START = "com.example.research.START_DOWNLOAD" const val ACTION_START = "com.example.research.START_DOWNLOAD"
const val ACTION_STOP = "com.example.research.STOP_DOWNLOAD" const val ACTION_STOP = "com.example.research.STOP_DOWNLOAD"
const val ACTION_IMPORT = "com.example.research.START_IMPORT" const val ACTION_IMPORT = "com.example.research.START_IMPORT"
const val ACTION_FINISH = "com.example.research.FINISH_PIPELINE"
private const val NOTIFICATION_ID = NotificationHelper.NOTIFICATION_ID private const val NOTIFICATION_ID = NotificationHelper.NOTIFICATION_ID
} }
private lateinit var downloadManager: DownloadManager private lateinit var downloadManager: DownloadManager
private lateinit var localDictionaryRepository: LocalDictionaryRepository private lateinit var localDictionaryRepository: LocalDictionaryRepository
private lateinit var dictionaryImportManager: com.example.research.feature.import.DictionaryImportManager private lateinit var dictionaryImportManager: com.example.research.feature.import.DictionaryImportManager
private lateinit var progressStateHolder: com.example.research.common.progress.DictionaryProgressStateHolder private lateinit var pipelineCoordinator: DictionaryPipelineCoordinator
private lateinit var presenter: DictionaryProgressPresenter
private val notificationHelper by lazy { private val notificationHelper by lazy {
NotificationHelper(this) NotificationHelper(this)
} }
private val serviceScope = CoroutineScope(SupervisorJob() + Dispatchers.Main) private val serviceScope = CoroutineScope(SupervisorJob() + Dispatchers.Main)
@Volatile @Volatile
private var isHandlingSuccess = false
@Volatile
private var latestStartId = 0 private var latestStartId = 0
override fun onCreate() { override fun onCreate() {
@@ -47,164 +47,8 @@ class DictionaryForegroundService : Service() {
downloadManager = app.downloadManager downloadManager = app.downloadManager
localDictionaryRepository = app.localDictionaryRepository localDictionaryRepository = app.localDictionaryRepository
dictionaryImportManager = app.dictionaryImportManager dictionaryImportManager = app.dictionaryImportManager
progressStateHolder = app.dictionaryProgressStateHolder pipelineCoordinator = app.dictionaryPipelineCoordinator
observeDownloadState() presenter = app.dictionaryProgressPresenter
observeImportState()
observeUnifiedProgress()
}
private fun observeDownloadState() {
serviceScope.launch {
downloadManager.downloadState.collect { state ->
when (state) {
is DownloadState.Success -> handleSuccess()
is DownloadState.Error -> {
notificationHelper.showErrorNotification()
stopForegroundService(removeNotification = false)
}
is DownloadState.Cancelled -> {
notificationHelper.cancelNotification()
stopForegroundService()
}
else -> {}
}
}
}
}
private fun observeImportState() {
serviceScope.launch {
dictionaryImportManager.importState.collect { state ->
when (state) {
is com.example.research.ui.settings.ImportState.Success -> handleSuccess()
is com.example.research.ui.settings.ImportState.Error -> {
notificationHelper.showErrorNotification()
stopForegroundService(removeNotification = false)
}
else -> {}
}
}
}
}
private fun observeUnifiedProgress() {
serviceScope.launch {
progressStateHolder.progressSnapshot
.collect { snapshot ->
if (snapshot != null) {
notificationHelper.showUnifiedProgressNotification(
title = snapshot.renderTitle(this@DictionaryForegroundService),
contentText = "${snapshot.percent}%",
progressPercent = snapshot.percent,
)
}
}
}
}
private suspend fun triggerReindexing(isImportFlow: Boolean) = withContext(Dispatchers.IO) {
try {
waitForIndexingCompletion()
delay(1.seconds)
val app = application as? ReSearchApplication
?: return@withContext
val path = app.preferencesManager.dictionaryPath
if (isImportFlow) {
dictionaryImportManager.updateExtractionProgress(0f)
}
app.downloadDictionaryRepository.extractArchives(onProgress = { progress ->
if (isImportFlow) {
dictionaryImportManager.updateExtractionProgress(progress)
}
})
val result = localDictionaryRepository.scanDirectory(path)
if (result is OperationResult.Success && result.data > 0) {
localDictionaryRepository.warmupIndexes()
app.downloadDictionaryRepository.performAllCleanup()
}
if (isImportFlow) {
dictionaryImportManager.markImportPipelineSuccess()
}
} catch (e: CancellationException) {
throw e
} catch (e: Exception) {
Log.e("DictionaryForegroundService", "triggerReindexing() failed: ${e.message}", e)
}
}
private suspend fun waitForIndexingCompletion(): Boolean {
val isIndexing = localDictionaryRepository.indexingProgress.first().isIndexing
if (!isIndexing) {
return true
}
var waitCount = 0
while (localDictionaryRepository.indexingProgress.first().isIndexing && waitCount < 100) {
delay(200.milliseconds)
waitCount++
}
return waitCount < 100
}
private fun handleSuccess() {
if (isHandlingSuccess) return
isHandlingSuccess = true
serviceScope.launch {
try {
val isImportFlow = dictionaryImportManager.importState.value !is com.example.research.ui.settings.ImportState.Idle
if (isImportFlow) {
val app = application as? ReSearchApplication
val path = app?.preferencesManager?.dictionaryPath
val dictionariesDir = path?.let { java.io.File(it) }
val filesBeforeReindex = dictionariesDir?.listFiles()?.map { it.name }?.toSet() ?: emptySet()
try {
triggerReindexing(true)
dictionaryImportManager.getAndClearImportedFiles()
} catch (e: CancellationException) {
val filesToCleanup = dictionaryImportManager.getAndClearImportedFiles()
filesToCleanup.forEach { file ->
try { if (file.exists()) file.delete() } catch (_: Exception) { /* Ignore cleanup failure */ }
}
if (dictionariesDir != null) {
cleanupNewFiles(dictionariesDir, filesBeforeReindex)
}
throw e
}
} else {
triggerReindexing(false)
}
if (isImportFlow) {
notificationHelper.showImportSuccessNotification()
dictionaryImportManager.clearImportState()
} else {
notificationHelper.showSuccessNotification()
}
} catch (e: Exception) {
if (e !is CancellationException) {
Log.e("DownloadForegroundService", "handleSuccess() failed: ${e.message}", e)
notificationHelper.showErrorNotification()
}
} finally {
isHandlingSuccess = false
stopForegroundService(removeNotification = false)
}
}
}
private fun cleanupNewFiles(dir: java.io.File, filesBeforeSnapshot: Set<String>) {
try {
dir.listFiles()?.forEach { file ->
if (file.name !in filesBeforeSnapshot) {
try { file.delete() } catch (_: Exception) { /* Ignore delete failure */ }
}
}
} catch (_: Exception) { /* Ignore directory listing failure */ }
} }
private fun startForegroundService() { private fun startForegroundService() {
@@ -249,15 +93,20 @@ class DictionaryForegroundService : Service() {
} }
private fun stopForTimeout(startId: Int) { private fun stopForTimeout(startId: Int) {
cancelAllWork()
ServiceCompat.stopForeground(this, ServiceCompat.STOP_FOREGROUND_REMOVE)
stopSelfResult(startId)
}
private fun cancelAllWork() {
serviceScope.coroutineContext[Job]?.cancelChildren() serviceScope.coroutineContext[Job]?.cancelChildren()
pipelineCoordinator.cancelPipeline()
if (localDictionaryRepository.isIndexingInProgress()) { if (localDictionaryRepository.isIndexingInProgress()) {
localDictionaryRepository.cancelIndexing() localDictionaryRepository.cancelIndexing()
} }
downloadManager.cancelDownload() downloadManager.cancelDownload()
dictionaryImportManager.cancelImport() dictionaryImportManager.cancelImport()
notificationHelper.cancelNotification() presenter.cancelProgress()
ServiceCompat.stopForeground(this, ServiceCompat.STOP_FOREGROUND_REMOVE)
stopSelfResult(startId)
} }
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int { override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
@@ -275,21 +124,16 @@ class DictionaryForegroundService : Service() {
ACTION_IMPORT -> { ACTION_IMPORT -> {
startForegroundService() startForegroundService()
} }
ACTION_FINISH -> {
stopForegroundService(removeNotification = false)
}
ACTION_STOP -> { ACTION_STOP -> {
try { try {
startForegroundService() startForegroundService()
} catch (_: Exception) { } catch (_: Exception) {
// Service may already be stopped // Service may already be stopped
} }
serviceScope.coroutineContext[Job]?.cancelChildren() cancelAllWork()
if (localDictionaryRepository.isIndexingInProgress()) {
localDictionaryRepository.cancelIndexing()
}
downloadManager.cancelDownload()
dictionaryImportManager.cancelImport()
notificationHelper.cancelNotification()
stopForegroundService() stopForegroundService()
} }
} }
@@ -3,6 +3,7 @@ package com.example.research.feature.import
import android.app.Application import android.app.Application
import android.net.Uri import android.net.Uri
import com.example.research.R import com.example.research.R
import com.example.research.common.progress.ImportFlowOperations
import com.example.research.common.util.SafeFileName import com.example.research.common.util.SafeFileName
import com.example.research.core.performance.ReSearchTrace import com.example.research.core.performance.ReSearchTrace
import com.example.research.ui.settings.ImportState import com.example.research.ui.settings.ImportState
@@ -14,16 +15,19 @@ import java.util.Collections
class DictionaryImportManager( class DictionaryImportManager(
private val application: Application private val application: Application
) { ) : ImportFlowOperations {
private val mutableImportState = MutableStateFlow<ImportState>(ImportState.Idle) private val mutableImportState = MutableStateFlow<ImportState>(ImportState.Idle)
val importState: StateFlow<ImportState> = mutableImportState.asStateFlow() val importState: StateFlow<ImportState> = mutableImportState.asStateFlow()
private val managerScope = CoroutineScope(SupervisorJob() + Dispatchers.Main) private val managerScope = CoroutineScope(SupervisorJob() + Dispatchers.Main)
private var importJob: Job? = null private var importJob: Job? = null
private val importedFiles = Collections.synchronizedList(mutableListOf<File>()) private val importedFiles = Collections.synchronizedList(mutableListOf<File>())
var onFlowStarted: (() -> Unit)? = null
var onTerminal: ((ImportState) -> Unit)? = null
fun importDictionaries(uris: List<Uri>) { fun importDictionaries(uris: List<Uri>) {
importJob?.cancel() importJob?.cancel()
onFlowStarted?.invoke()
importJob = managerScope.launch { importJob = managerScope.launch {
mutableImportState.value = ImportState.Idle mutableImportState.value = ImportState.Idle
performImport(uris) performImport(uris)
@@ -36,6 +40,8 @@ class DictionaryImportManager(
val dictionariesDir = File(context.getExternalFilesDir(null), "dictionaries") val dictionariesDir = File(context.getExternalFilesDir(null), "dictionaries")
val totalFiles = uris.size.coerceAtLeast(1) val totalFiles = uris.size.coerceAtLeast(1)
importedFiles.clear() importedFiles.clear()
val skippedNames = mutableListOf<String>()
val deferredErrors = mutableListOf<String>()
if (!dictionariesDir.exists()) { if (!dictionariesDir.exists()) {
dictionariesDir.mkdirs() dictionariesDir.mkdirs()
@@ -46,9 +52,7 @@ class DictionaryImportManager(
currentCoroutineContext().ensureActive() currentCoroutineContext().ensureActive()
val fileName = getFileName(uri) ?: continue val fileName = getFileName(uri) ?: continue
if (SafeFileName.validate(fileName) == null) { if (SafeFileName.validate(fileName) == null) {
mutableImportState.value = ImportState.Error( deferredErrors += context.getString(R.string.import_invalid_file_name, fileName)
context.getString(R.string.import_invalid_file_name, fileName)
)
continue continue
} }
val lowerFileName = fileName.lowercase() val lowerFileName = fileName.lowercase()
@@ -65,9 +69,7 @@ class DictionaryImportManager(
val destFile = File(dictionariesDir, fileName) val destFile = File(dictionariesDir, fileName)
if (destFile.exists()) { if (destFile.exists()) {
mutableImportState.value = ImportState.Error( skippedNames += fileName
context.getString(R.string.import_file_exists, fileName)
)
continue continue
} }
@@ -115,16 +117,30 @@ class DictionaryImportManager(
currentCoroutineContext().ensureActive() currentCoroutineContext().ensureActive()
mutableImportState.value = ImportState.Success val terminalState = resolveImportOutcome(
importedCount = importedFiles.size,
skippedNames = skippedNames,
deferredErrors = deferredErrors,
).toImportState(
existsMessage = { names ->
context.getString(R.string.import_file_exists, names.joinToString(separator = ", "))
},
invalidMessage = { message -> message },
nothingImportedMessage = { context.getString(R.string.import_nothing_imported) },
)
mutableImportState.value = terminalState
onTerminal?.invoke(terminalState)
} catch (e: CancellationException) { } catch (e: CancellationException) {
cleanupImportedFiles() cleanupImportedFiles()
mutableImportState.value = ImportState.Idle mutableImportState.value = ImportState.Idle
throw e throw e
} catch (e: Exception) { } catch (e: Exception) {
mutableImportState.value = ImportState.Error( val errorState = ImportState.Error(
context.getString(R.string.import_error, e.message ?: "Unknown error") context.getString(R.string.import_error, e.message ?: "Unknown error")
) )
mutableImportState.value = errorState
onTerminal?.invoke(errorState)
} }
} }
@@ -154,18 +170,18 @@ class DictionaryImportManager(
return size return size
} }
fun clearImportState() { override fun clearImportState() {
mutableImportState.value = ImportState.Idle mutableImportState.value = ImportState.Idle
} }
fun updateExtractionProgress(progress: Float) { override fun updateExtractionProgress(progress: Float) {
mutableImportState.update { current -> mutableImportState.update { current ->
if (current is ImportState.Idle || current is ImportState.Error) current if (current is ImportState.Idle || current is ImportState.Error) current
else ImportState.Extracting(progress.coerceIn(0f, 1f)) else ImportState.Extracting(progress.coerceIn(0f, 1f))
} }
} }
fun markImportPipelineSuccess() { override fun markImportPipelineSuccess() {
mutableImportState.update { current -> mutableImportState.update { current ->
if (current is ImportState.Idle || current is ImportState.Error) current if (current is ImportState.Idle || current is ImportState.Error) current
else ImportState.Success else ImportState.Success
@@ -179,7 +195,7 @@ class DictionaryImportManager(
mutableImportState.value = ImportState.Idle mutableImportState.value = ImportState.Idle
} }
fun getAndClearImportedFiles(): List<File> { override fun getAndClearImportedFiles(): List<File> {
val files = importedFiles.toList() val files = importedFiles.toList()
importedFiles.clear() importedFiles.clear()
return files return files
@@ -0,0 +1,32 @@
package com.example.research.feature.import
import com.example.research.ui.settings.ImportState
internal sealed interface ImportOutcome {
data object Imported : ImportOutcome
data class AlreadyExists(val names: List<String>) : ImportOutcome
data class InvalidFile(val message: String) : ImportOutcome
data object NothingImported : ImportOutcome
}
internal fun resolveImportOutcome(
importedCount: Int,
skippedNames: List<String>,
deferredErrors: List<String>,
): ImportOutcome = when {
importedCount > 0 -> ImportOutcome.Imported
skippedNames.isNotEmpty() -> ImportOutcome.AlreadyExists(skippedNames)
deferredErrors.isNotEmpty() -> ImportOutcome.InvalidFile(deferredErrors.first())
else -> ImportOutcome.NothingImported
}
internal fun ImportOutcome.toImportState(
existsMessage: (List<String>) -> String,
invalidMessage: (String) -> String,
nothingImportedMessage: () -> String,
): ImportState = when (this) {
ImportOutcome.Imported -> ImportState.Success
is ImportOutcome.AlreadyExists -> ImportState.Error(existsMessage(names))
is ImportOutcome.InvalidFile -> ImportState.Error(invalidMessage(message))
ImportOutcome.NothingImported -> ImportState.Error(nothingImportedMessage())
}
@@ -85,7 +85,12 @@ class SearchViewModel(
@OptIn(FlowPreview::class, ExperimentalCoroutinesApi::class) @OptIn(FlowPreview::class, ExperimentalCoroutinesApi::class)
val searchResults: Flow<PagingData<IndexEntry>> = combine( val searchResults: Flow<PagingData<IndexEntry>> = combine(
searchQuery.debounce(300.milliseconds).distinctUntilChanged(), searchQuery
.map { it to it.isBlank() }
.distinctUntilChangedBy { it.first }
.flatMapLatest { (query, isBlank) ->
if (isBlank) flowOf(query) else flowOf(query).debounce(300.milliseconds)
},
activeDictionaries activeDictionaries
) { query, dictionaries -> ) { query, dictionaries ->
query to dictionaries query to dictionaries
@@ -98,8 +103,7 @@ class SearchViewModel(
pageSize = 12, pageSize = 12,
prefetchDistance = 1, prefetchDistance = 1,
enablePlaceholders = false, enablePlaceholders = false,
initialLoadSize = 12, initialLoadSize = 12
jumpThreshold = 1
), ),
pagingSourceFactory = { pagingSourceFactory = {
IndexEntryPagingSource( IndexEntryPagingSource(
@@ -78,6 +78,4 @@ internal object AboutLibrariesParser {
context.resources.openRawResource(resourceId) context.resources.openRawResource(resourceId)
.bufferedReader() .bufferedReader()
.use { reader -> json.decodeFromString(reader.readText()) } .use { reader -> json.decodeFromString(reader.readText()) }
fun decode(source: String): AboutLibrariesData = json.decodeFromString(source)
} }
@@ -1,6 +1,5 @@
package com.example.research.ui.about package com.example.research.ui.about
import androidx.compose.animation.animateContentSize
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ExperimentalLayoutApi import androidx.compose.foundation.layout.ExperimentalLayoutApi
@@ -27,6 +26,7 @@ import androidx.compose.material3.TopAppBarDefaults
import androidx.compose.runtime.Composable import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.produceState
import androidx.compose.runtime.remember import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment import androidx.compose.ui.Alignment
@@ -43,6 +43,8 @@ import androidx.compose.ui.unit.dp
import com.example.research.R import com.example.research.R
import com.example.research.common.ui.components.OutlinedChoiceButton import com.example.research.common.ui.components.OutlinedChoiceButton
import com.example.research.ui.theme.AppWindowInsets import com.example.research.ui.theme.AppWindowInsets
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
@OptIn(ExperimentalMaterial3Api::class, ExperimentalLayoutApi::class) @OptIn(ExperimentalMaterial3Api::class, ExperimentalLayoutApi::class)
@Composable @Composable
@@ -57,9 +59,11 @@ fun AboutScreen(
} }
val versionName = packageInfo?.versionName ?: stringResource(R.string.version_unknown) val versionName = packageInfo?.versionName ?: stringResource(R.string.version_unknown)
val uriHandler = LocalUriHandler.current val uriHandler = LocalUriHandler.current
val librariesData = remember(context) { val librariesData by produceState<AboutLibrariesData?>(initialValue = null, context) {
value = withContext(Dispatchers.IO) {
AboutLibrariesParser.read(context, R.raw.aboutlibraries) AboutLibrariesParser.read(context, R.raw.aboutlibraries)
} }
}
var expandedLibraryId by remember { mutableStateOf<String?>(null) } var expandedLibraryId by remember { mutableStateOf<String?>(null) }
var dialogLicense by remember { mutableStateOf<AboutLicense?>(null) } var dialogLicense by remember { mutableStateOf<AboutLicense?>(null) }
@@ -93,12 +97,12 @@ fun AboutScreen(
AboutHeader(versionName = versionName) AboutHeader(versionName = versionName)
} }
items( items(
items = librariesData.libraries, items = librariesData?.libraries.orEmpty(),
key = AboutLibrary::uniqueId, key = AboutLibrary::uniqueId,
) { library -> ) { library ->
AboutLibraryRow( AboutLibraryRow(
library = library, library = library,
licenses = librariesData.licenses, licenses = librariesData?.licenses.orEmpty(),
expanded = expandedLibraryId == library.uniqueId, expanded = expandedLibraryId == library.uniqueId,
onToggle = { onToggle = {
expandedLibraryId = if (expandedLibraryId == library.uniqueId) { expandedLibraryId = if (expandedLibraryId == library.uniqueId) {
@@ -191,8 +195,7 @@ private fun AboutLibraryRow(
) { ) {
Column( Column(
modifier = Modifier modifier = Modifier
.padding(horizontal = 16.dp, vertical = 12.dp) .padding(horizontal = 16.dp, vertical = 12.dp),
.animateContentSize(),
verticalArrangement = Arrangement.spacedBy(5.dp) verticalArrangement = Arrangement.spacedBy(5.dp)
) { ) {
Row( Row(
@@ -22,6 +22,7 @@ import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.IntSize import androidx.compose.ui.unit.IntSize
import androidx.lifecycle.compose.collectAsStateWithLifecycle import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.example.research.R import com.example.research.R
import com.example.research.common.ui.theme.dictionaryTitleLarge
import com.example.research.feature.search.SearchAction import com.example.research.feature.search.SearchAction
import com.example.research.ui.theme.Spacing import com.example.research.ui.theme.Spacing
import com.example.research.ui.theme.AppWindowInsets import com.example.research.ui.theme.AppWindowInsets
@@ -258,7 +259,7 @@ fun ArticleTitleBar(
Text( Text(
text = title, text = title,
style = MaterialTheme.typography.titleLarge, style = MaterialTheme.typography.dictionaryTitleLarge,
color = MaterialTheme.colorScheme.onSurface, color = MaterialTheme.colorScheme.onSurface,
maxLines = 1, maxLines = 1,
overflow = TextOverflow.Ellipsis, overflow = TextOverflow.Ellipsis,
@@ -23,7 +23,6 @@ object DslAnnotatedParser {
"br", "p", "b", "i", "c", "t", "m", "m0", "m1", "m2", "m3", "m4", "m5", "br", "p", "b", "i", "c", "t", "m", "m0", "m1", "m2", "m3", "m4", "m5",
"ref", "ex", "e", "trn", "com", "lang", "sup", "'" "ref", "ex", "e", "trn", "com", "lang", "sup", "'"
) )
private val ESCAPED_CHARS = setOf('[', ']', '(', ')')
data class ColorScheme( data class ColorScheme(
val secondaryText: Color, val secondaryText: Color,
@@ -101,7 +100,6 @@ object DslAnnotatedParser {
'\\' -> { '\\' -> {
if (i + 1 < length) { if (i + 1 < length) {
val next = dsl[i + 1] val next = dsl[i + 1]
if (next in ESCAPED_CHARS) {
builder.append(next) builder.append(next)
if (refStack.isNotEmpty()) { if (refStack.isNotEmpty()) {
refStack.last().second.append(next) refStack.last().second.append(next)
@@ -110,7 +108,6 @@ object DslAnnotatedParser {
i += 2 i += 2
continue continue
} }
}
builder.append(char) builder.append(char)
if (refStack.isNotEmpty()) { if (refStack.isNotEmpty()) {
refStack.last().second.append(char) refStack.last().second.append(char)
@@ -119,7 +116,7 @@ object DslAnnotatedParser {
i++ i++
} }
'[' -> { '[' -> {
val end = dsl.indexOf(']', i + 1) val end = tagCloseIndex(dsl, i + 1)
if (end != -1) { if (end != -1) {
val tagStart = i + 1 val tagStart = i + 1
var tagEnd = end var tagEnd = end
@@ -228,6 +225,19 @@ object DslAnnotatedParser {
splitOversizedBlocks(result) splitOversizedBlocks(result)
} }
private fun tagCloseIndex(value: String, startIndex: Int): Int {
var index = startIndex
while (index < value.length) {
when {
value[index] == '\\' && index + 1 < value.length -> index += 2
value[index] == '[' -> return -1
value[index] == ']' -> return index
else -> index++
}
}
return -1
}
private fun splitOversizedBlocks(blocks: List<DslBlock>): List<DslBlock> { private fun splitOversizedBlocks(blocks: List<DslBlock>): List<DslBlock> {
if (blocks.none { it.text.length > MAX_BLOCK_TEXT_LENGTH }) return blocks if (blocks.none { it.text.length > MAX_BLOCK_TEXT_LENGTH }) return blocks
return blocks.flatMap { block -> return blocks.flatMap { block ->
@@ -1,24 +1,26 @@
package com.example.research.ui.main.components package com.example.research.ui.main.components
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.offset
import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.windowInsetsPadding import androidx.compose.foundation.layout.windowInsetsPadding
import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.text.BasicTextField
import androidx.compose.foundation.text.KeyboardOptions import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.foundation.text.input.TextFieldLineLimits import androidx.compose.foundation.text.input.TextFieldLineLimits
import androidx.compose.foundation.text.input.TextFieldState import androidx.compose.foundation.text.input.TextFieldState
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.FilledIconButton import androidx.compose.material3.FilledIconButton
import androidx.compose.material3.Icon import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton import androidx.compose.material3.IconButton
import androidx.compose.material3.IconButtonDefaults import androidx.compose.material3.IconButtonDefaults
import androidx.compose.material3.MaterialTheme import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface import androidx.compose.material3.Surface
import androidx.compose.material3.TextField
import androidx.compose.material3.TextFieldDefaults
import androidx.compose.runtime.Composable import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue import androidx.compose.runtime.getValue
@@ -27,7 +29,7 @@ import androidx.compose.runtime.snapshotFlow
import com.example.research.ui.theme.AppWindowInsets import com.example.research.ui.theme.AppWindowInsets
import androidx.compose.ui.Alignment import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.SolidColor import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.testTag import androidx.compose.ui.platform.testTag
import androidx.compose.ui.res.painterResource import androidx.compose.ui.res.painterResource
import androidx.compose.ui.res.stringResource import androidx.compose.ui.res.stringResource
@@ -36,9 +38,11 @@ import androidx.compose.ui.semantics.testTagsAsResourceId
import androidx.compose.ui.text.input.ImeAction import androidx.compose.ui.text.input.ImeAction
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import com.example.research.R import com.example.research.R
import com.example.research.common.ui.theme.dictionaryTitleLarge
import kotlinx.coroutines.flow.distinctUntilChanged import kotlinx.coroutines.flow.distinctUntilChanged
import com.example.research.ui.theme.Spacing
private val TopBarHeight = 56.dp private val TopBarHeight = Spacing.topBarHeight
private val TopBarShape = RoundedCornerShape(28.dp) private val TopBarShape = RoundedCornerShape(28.dp)
@Composable @Composable
@@ -99,6 +103,7 @@ fun MainTopBar(
} }
@Composable @Composable
@OptIn(ExperimentalMaterial3Api::class)
private fun SearchField( private fun SearchField(
searchState: TextFieldState, searchState: TextFieldState,
enabled: Boolean, enabled: Boolean,
@@ -110,8 +115,6 @@ private fun SearchField(
modifier: Modifier = Modifier modifier: Modifier = Modifier
) { ) {
val containerColor = MaterialTheme.colorScheme.surfaceContainerHighest val containerColor = MaterialTheme.colorScheme.surfaceContainerHighest
val contentColor = MaterialTheme.colorScheme.onSurface
val placeholderColor = MaterialTheme.colorScheme.onSurfaceVariant
val currentOnQueryChange by rememberUpdatedState(onQueryChange) val currentOnQueryChange by rememberUpdatedState(onQueryChange)
@@ -123,57 +126,60 @@ private fun SearchField(
} }
} }
Surface( TextField(
modifier = modifier.height(TopBarHeight), state = searchState,
color = containerColor, enabled = enabled,
shape = TopBarShape textStyle = MaterialTheme.typography.dictionaryTitleLarge,
) { keyboardOptions = KeyboardOptions(imeAction = ImeAction.Search),
Row( lineLimits = TextFieldLineLimits.SingleLine,
modifier = Modifier.padding(horizontal = 16.dp), leadingIcon = {
verticalAlignment = Alignment.CenterVertically
) {
Icon( Icon(
painter = searchIcon, painter = searchIcon,
contentDescription = null, contentDescription = null,
tint = placeholderColor,
modifier = Modifier.size(24.dp)
)
BasicTextField(
state = searchState,
enabled = enabled,
textStyle = MaterialTheme.typography.titleLarge.copy(color = contentColor),
cursorBrush = SolidColor(MaterialTheme.colorScheme.primary),
keyboardOptions = KeyboardOptions(imeAction = ImeAction.Search),
lineLimits = TextFieldLineLimits.SingleLine,
modifier = Modifier modifier = Modifier
.weight(1f) .offset(x = 4.dp)
.padding(horizontal = 12.dp) .size(24.dp)
.testTag("search_field")
.semantics { testTagsAsResourceId = true }
) )
},
if (searchState.text.isNotEmpty()) { trailingIcon = if (searchState.text.isNotEmpty()) {
{
IconButton( IconButton(
onClick = { onClick = {
searchState.edit { replace(0, length, "") } searchState.edit { replace(0, length, "") }
onClearQuery() onClearQuery()
}, },
modifier = Modifier modifier = Modifier
.size(40.dp) .padding(end = 12.dp)
.testTag("clear_search") .testTag("clear_search")
.semantics { testTagsAsResourceId = true } .semantics { testTagsAsResourceId = true }
) { ) {
Icon( Icon(
painter = closeIcon, painter = closeIcon,
contentDescription = clearLabel, contentDescription = clearLabel,
tint = placeholderColor,
modifier = Modifier.size(24.dp) modifier = Modifier.size(24.dp)
) )
} }
}
} else { } else {
Box(modifier = Modifier.size(40.dp)) null
} },
} shape = TopBarShape,
} contentPadding = TextFieldDefaults.contentPaddingWithoutLabel(
top = 8.dp,
bottom = 8.dp
),
colors = TextFieldDefaults.colors(
focusedContainerColor = containerColor,
unfocusedContainerColor = containerColor,
disabledContainerColor = containerColor,
cursorColor = MaterialTheme.colorScheme.primary,
focusedIndicatorColor = Color.Transparent,
unfocusedIndicatorColor = Color.Transparent,
disabledIndicatorColor = Color.Transparent
),
modifier = modifier
.height(TopBarHeight)
.testTag("search_field")
.semantics { testTagsAsResourceId = true }
)
} }
@@ -17,6 +17,7 @@ import androidx.compose.ui.unit.dp
import androidx.paging.LoadState import androidx.paging.LoadState
import androidx.paging.compose.* import androidx.paging.compose.*
import com.example.research.R import com.example.research.R
import com.example.research.common.ui.theme.dictionaryTitleLarge
import com.example.research.core.domain.model.IndexEntry import com.example.research.core.domain.model.IndexEntry
import com.example.research.core.util.removeDictionarySuffixes import com.example.research.core.util.removeDictionarySuffixes
import com.example.research.ui.theme.Spacing import com.example.research.ui.theme.Spacing
@@ -40,7 +41,7 @@ fun SearchResultsList(
) { ) {
items( items(
count = results.itemCount, count = results.itemCount,
key = results.itemKey { entry -> "${entry.dictionaryPath}_${entry.offset.value}_${entry.word}" }, key = results.itemKey { "${it.dictionaryPath}:${it.offset.value}:${it.word}" },
contentType = results.itemContentType { "search_result" } contentType = results.itemContentType { "search_result" }
) { index -> ) { index ->
results[index]?.let { entry -> results[index]?.let { entry ->
@@ -83,7 +84,7 @@ private fun SearchResultItem(
showDictionaryName: Boolean = true showDictionaryName: Boolean = true
) { ) {
// Cache text styles to avoid recreation on every composition // Cache text styles to avoid recreation on every composition
val titleStyle = MaterialTheme.typography.titleLarge val titleStyle = MaterialTheme.typography.dictionaryTitleLarge
val subtitleStyle = MaterialTheme.typography.bodySmall val subtitleStyle = MaterialTheme.typography.bodySmall
val subtitleColor = MaterialTheme.colorScheme.onSurfaceVariant val subtitleColor = MaterialTheme.colorScheme.onSurfaceVariant
@@ -2,22 +2,18 @@ package com.example.research.ui.navigation
import androidx.activity.compose.BackHandler import androidx.activity.compose.BackHandler
import androidx.compose.material3.adaptive.ExperimentalMaterial3AdaptiveApi import androidx.compose.material3.adaptive.ExperimentalMaterial3AdaptiveApi
import androidx.compose.material3.adaptive.currentWindowAdaptiveInfo import androidx.compose.material3.adaptive.currentWindowAdaptiveInfoV2
import androidx.compose.runtime.* import androidx.compose.runtime.*
import androidx.compose.runtime.saveable.* import androidx.compose.runtime.saveable.*
import androidx.lifecycle.compose.collectAsStateWithLifecycle import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.window.core.layout.WindowSizeClass.Companion.WIDTH_DP_MEDIUM_LOWER_BOUND import androidx.window.core.layout.WindowSizeClass.Companion.WIDTH_DP_MEDIUM_LOWER_BOUND
import com.example.research.feature.search.SearchAction import com.example.research.feature.search.SearchAction
import com.example.research.feature.search.SearchViewModel import com.example.research.feature.search.SearchViewModel
import com.example.research.feature.download.model.DownloadState
import com.example.research.ui.about.AboutScreen import com.example.research.ui.about.AboutScreen
import com.example.research.ui.article.ArticleRoute import com.example.research.ui.article.ArticleRoute
import com.example.research.ui.main.MainRoute import com.example.research.ui.main.MainRoute
import com.example.research.ui.settings.ImportState
import com.example.research.ui.settings.SettingsRoute import com.example.research.ui.settings.SettingsRoute
import com.example.research.ui.settings.SettingsViewModel import com.example.research.ui.settings.SettingsViewModel
import kotlinx.coroutines.delay
import kotlin.time.Duration.Companion.milliseconds
private data class NavigationState( private data class NavigationState(
val shouldShowSettings: Boolean, val shouldShowSettings: Boolean,
@@ -33,7 +29,8 @@ enum class Screen {
@Composable @Composable
fun AppNavigation( fun AppNavigation(
searchViewModel: SearchViewModel, searchViewModel: SearchViewModel,
settingsViewModel: SettingsViewModel settingsViewModel: SettingsViewModel,
seedNoDictionaries: Boolean = false
) { ) {
val screenStack = rememberSaveable( val screenStack = rememberSaveable(
saver = listSaver( saver = listSaver(
@@ -53,39 +50,15 @@ fun AppNavigation(
val currentScreen = screenStack.lastOrNull() ?: Screen.Home val currentScreen = screenStack.lastOrNull() ?: Screen.Home
val settingsState by settingsViewModel.uiState.collectAsStateWithLifecycle() val settingsState by settingsViewModel.uiState.collectAsStateWithLifecycle()
val progressSnapshot by settingsViewModel.progressSnapshot.collectAsStateWithLifecycle()
val isPipelineActiveByState = remember(
progressSnapshot,
settingsState.indexingProgress.isIndexing,
settingsState.downloadState,
settingsState.importState,
) {
progressSnapshot != null ||
settingsState.indexingProgress.isIndexing ||
settingsState.downloadState is DownloadState.Loading ||
settingsState.downloadState is DownloadState.Extracting ||
settingsState.downloadState is DownloadState.Success ||
settingsState.importState is ImportState.Importing ||
settingsState.importState is ImportState.Extracting ||
settingsState.importState is ImportState.Success
}
var stickyPipelineActive by rememberSaveable { mutableStateOf(false) }
LaunchedEffect(isPipelineActiveByState) {
if (isPipelineActiveByState) {
stickyPipelineActive = true
} else {
delay(1500.milliseconds)
stickyPipelineActive = false
}
}
val navState by remember { val navState by remember {
derivedStateOf { derivedStateOf {
val isOperationActive = stickyPipelineActive val isOperationActive = !settingsState.pipelineIdle
val hasConfirmedNoDictionaries = val hasConfirmedNoDictionaries =
settingsState.hasCompletedStartupScan && settingsState.dictionaries.isEmpty() settingsState.hasCompletedStartupScan && settingsState.dictionaries.isEmpty()
val hasSeededNoDictionaries = seedNoDictionaries && !settingsState.hasCompletedStartupScan
NavigationState( NavigationState(
shouldShowSettings = hasConfirmedNoDictionaries || isOperationActive, shouldShowSettings = hasConfirmedNoDictionaries || hasSeededNoDictionaries || isOperationActive,
showBackButtonInSettings = settingsState.dictionaries.isNotEmpty() && !isOperationActive, showBackButtonInSettings = settingsState.dictionaries.isNotEmpty() && !isOperationActive,
isOperationActive = isOperationActive, isOperationActive = isOperationActive,
) )
@@ -95,7 +68,7 @@ fun AppNavigation(
val shouldShowSettings = navState.shouldShowSettings val shouldShowSettings = navState.shouldShowSettings
val showBackButtonInSettings = navState.showBackButtonInSettings val showBackButtonInSettings = navState.showBackButtonInSettings
val isOperationActive = navState.isOperationActive val isOperationActive = navState.isOperationActive
val isWideScreen = currentWindowAdaptiveInfo() val isWideScreen = currentWindowAdaptiveInfoV2()
.windowSizeClass .windowSizeClass
.isWidthAtLeastBreakpoint(WIDTH_DP_MEDIUM_LOWER_BOUND) .isWidthAtLeastBreakpoint(WIDTH_DP_MEDIUM_LOWER_BOUND)
@@ -1,5 +1,8 @@
package com.example.research.ui.settings package com.example.research.ui.settings
import androidx.compose.runtime.Immutable
@Immutable
sealed interface ImportState { sealed interface ImportState {
data object Idle : ImportState data object Idle : ImportState
data class Importing(val progress: Float) : ImportState data class Importing(val progress: Float) : ImportState
@@ -7,3 +10,6 @@ sealed interface ImportState {
data object Success : ImportState data object Success : ImportState
data class Error(val message: String) : ImportState data class Error(val message: String) : ImportState
} }
val ImportState.isActive: Boolean
get() = this is ImportState.Importing || this is ImportState.Extracting || this is ImportState.Success
@@ -1,6 +1,7 @@
package com.example.research.ui.settings package com.example.research.ui.settings
import android.net.Uri import android.net.Uri
import androidx.compose.runtime.Immutable
import com.example.research.core.domain.model.AppTheme import com.example.research.core.domain.model.AppTheme
import com.example.research.core.domain.model.Dictionary import com.example.research.core.domain.model.Dictionary
import com.example.research.core.domain.model.DictionarySource import com.example.research.core.domain.model.DictionarySource
@@ -8,6 +9,7 @@ import com.example.research.core.domain.model.IndexingProgress
import com.example.research.DictionaryStatus import com.example.research.DictionaryStatus
import com.example.research.feature.download.model.DownloadState import com.example.research.feature.download.model.DownloadState
@Immutable
data class SettingsUiState( data class SettingsUiState(
val theme: AppTheme = AppTheme.SYSTEM, val theme: AppTheme = AppTheme.SYSTEM,
val language: String = "system", val language: String = "system",
@@ -21,6 +23,7 @@ data class SettingsUiState(
val importState: ImportState = ImportState.Idle, val importState: ImportState = ImportState.Idle,
val dictionarySources: List<DictionarySource> = emptyList(), val dictionarySources: List<DictionarySource> = emptyList(),
val hasCompletedStartupScan: Boolean = false, val hasCompletedStartupScan: Boolean = false,
val pipelineIdle: Boolean = true,
val appVersion: String = "1.0", val appVersion: String = "1.0",
) )
@@ -11,6 +11,7 @@ import com.example.research.core.domain.model.AppTheme
import com.example.research.core.domain.model.Dictionary import com.example.research.core.domain.model.Dictionary
import com.example.research.core.domain.model.DictionarySource import com.example.research.core.domain.model.DictionarySource
import com.example.research.core.domain.model.IndexingProgress import com.example.research.core.domain.model.IndexingProgress
import com.example.research.core.domain.usecase.DictionarySourceFileMatcher
import com.example.research.core.domain.usecase.DictionarySourceValidator import com.example.research.core.domain.usecase.DictionarySourceValidator
import com.example.research.core.domain.usecase.ManageDictionarySourcesUseCase import com.example.research.core.domain.usecase.ManageDictionarySourcesUseCase
import com.example.research.core.util.OperationResult import com.example.research.core.util.OperationResult
@@ -18,6 +19,7 @@ import com.example.research.data.local.preferences.PreferencesManager
import com.example.research.data.repository.LocalDictionaryRepository import com.example.research.data.repository.LocalDictionaryRepository
import com.example.research.feature.download.DownloadManager import com.example.research.feature.download.DownloadManager
import com.example.research.feature.download.model.DownloadState import com.example.research.feature.download.model.DownloadState
import com.example.research.feature.download.model.isActive
import com.example.research.feature.download.repository.DictionaryRepository import com.example.research.feature.download.repository.DictionaryRepository
import com.example.research.feature.import.DictionaryImportManager import com.example.research.feature.import.DictionaryImportManager
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
@@ -30,7 +32,7 @@ import kotlinx.coroutines.flow.map
import kotlinx.coroutines.flow.receiveAsFlow import kotlinx.coroutines.flow.receiveAsFlow
import kotlinx.coroutines.launch import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext import kotlinx.coroutines.withContext
import java.io.File import java.util.concurrent.ConcurrentHashMap
private data class DictionaryStateInputs( private data class DictionaryStateInputs(
val theme: AppTheme, val theme: AppTheme,
@@ -79,14 +81,29 @@ class SettingsViewModel(
private val language = MutableStateFlow("system") private val language = MutableStateFlow("system")
private val hasCompletedStartupScan = MutableStateFlow(false) private val hasCompletedStartupScan = MutableStateFlow(false)
private val pendingSourceUrls = mutableSetOf<String>() private val pendingSourceUrls = ConcurrentHashMap.newKeySet<String>()
private var isDownloadInProgress = false private var isDownloadInProgress = false
private var cancelRefreshPending = false
private var errorStateHandled = false
private var statusBeforeDownload: DictionaryStatus = DictionaryStatus.Unknown
init { init {
setupStateObservation() setupStateObservation()
observeDictionariesForSeedFlag()
initialize() initialize()
} }
private fun observeDictionariesForSeedFlag() {
viewModelScope.launch {
localDictionaryRepository.dictionaries.collect { dictionaries ->
if (hasCompletedStartupScan.value) {
preferencesManager.setHadNoDictionaries(dictionaries.isEmpty())
}
}
}
}
private fun setupStateObservation() { private fun setupStateObservation() {
viewModelScope.launch { viewModelScope.launch {
var wasIndexing = false var wasIndexing = false
@@ -143,31 +160,31 @@ class SettingsViewModel(
when (operationState.downloadState) { when (operationState.downloadState) {
is DownloadState.Loading, is DownloadState.Extracting -> { is DownloadState.Loading, is DownloadState.Extracting -> {
isDownloadInProgress = true isDownloadInProgress = true
cancelRefreshPending = false
errorStateHandled = false
} }
is DownloadState.Success -> { is DownloadState.Success -> {
pendingSourceUrls.clear() pendingSourceUrls.clear()
} }
is DownloadState.Error -> { is DownloadState.Error -> {
isDownloadInProgress = false isDownloadInProgress = false
if (!errorStateHandled) {
errorStateHandled = true
dictionaryStatus.value = if (dictionaryState.dictionaries.isEmpty()) { dictionaryStatus.value = if (dictionaryState.dictionaries.isEmpty()) {
DictionaryStatus.Empty DictionaryStatus.Empty
} else { } else when (statusBeforeDownload) {
DictionaryStatus.UpToDate DictionaryStatus.Unknown, DictionaryStatus.Checking -> DictionaryStatus.UpToDate
else -> statusBeforeDownload
} }
if (pendingSourceUrls.isNotEmpty()) { if (pendingSourceUrls.isNotEmpty()) {
pendingSourceUrls.forEach { urlTemplate -> pendingSourceUrls.forEach { urlTemplate ->
val source = dictionaryState.dictionarySources.find { val source = dictionaryState.dictionarySources.find {
it.urlTemplate == urlTemplate it.urlTemplate == urlTemplate
} }
if (source != null) { if (source != null) {
val hasDictionary = dictionaryState.dictionaries.any { dict -> val hasDictionary = dictionaryState.dictionaries.any { dictionary ->
DictionarySource.matchesDictionaryFile( DictionarySourceFileMatcher.matches(source, dictionary)
urlTemplate,
File(dict.path).name
)
} }
if (!hasDictionary) { if (!hasDictionary) {
manageDictionarySourcesUseCase.removeSource(source.id) manageDictionarySourcesUseCase.removeSource(source.id)
} }
@@ -176,13 +193,10 @@ class SettingsViewModel(
pendingSourceUrls.clear() pendingSourceUrls.clear()
} }
} }
}
is DownloadState.Cancelled -> { is DownloadState.Cancelled -> {
isDownloadInProgress = false isDownloadInProgress = false
dictionaryStatus.value = if (dictionaryState.dictionaries.isEmpty()) { cancelRefreshPending = true
DictionaryStatus.Empty
} else {
DictionaryStatus.UpToDate
}
if (pendingSourceUrls.isNotEmpty()) { if (pendingSourceUrls.isNotEmpty()) {
pendingSourceUrls.forEach { urlTemplate -> pendingSourceUrls.forEach { urlTemplate ->
val source = dictionaryState.dictionarySources.find { val source = dictionaryState.dictionarySources.find {
@@ -193,6 +207,12 @@ class SettingsViewModel(
pendingSourceUrls.clear() pendingSourceUrls.clear()
} }
} }
is DownloadState.Idle -> {
if (cancelRefreshPending) {
cancelRefreshPending = false
refreshDictionaryStatus()
}
}
else -> {} else -> {}
} }
@@ -206,6 +226,9 @@ class SettingsViewModel(
importState = operationState.importState, importState = operationState.importState,
dictionaryStatus = operationState.dictionaryStatus, dictionaryStatus = operationState.dictionaryStatus,
hasCompletedStartupScan = operationState.hasCompletedStartupScan, hasCompletedStartupScan = operationState.hasCompletedStartupScan,
pipelineIdle = !isIndexing &&
!operationState.downloadState.isActive &&
!operationState.importState.isActive,
isThemeExpanded = operationState.isThemeExpanded, isThemeExpanded = operationState.isThemeExpanded,
isLanguageExpanded = operationState.isLanguageExpanded, isLanguageExpanded = operationState.isLanguageExpanded,
isDictionariesExpanded = operationState.isDictionariesExpanded, isDictionariesExpanded = operationState.isDictionariesExpanded,
@@ -237,6 +260,9 @@ class SettingsViewModel(
android.util.Log.w("SettingsViewModel", "Failed to load language preference: ${e.message}") android.util.Log.w("SettingsViewModel", "Failed to load language preference: ${e.message}")
} }
hasCompletedStartupScan.value = true hasCompletedStartupScan.value = true
preferencesManager.setHadNoDictionaries(
localDictionaryRepository.dictionaries.value.isEmpty()
)
} }
} }
} }
@@ -259,18 +285,21 @@ class SettingsViewModel(
private suspend fun evaluateDictionaryStatus(): DictionaryStatus { private suspend fun evaluateDictionaryStatus(): DictionaryStatus {
return try { return try {
val actualDictionaries = localDictionaryRepository.dictionaries.first() val actualDictionaries = localDictionaryRepository.dictionaries.first()
val hasDictionaries = actualDictionaries.isNotEmpty()
if (!hasDictionaries) {
return DictionaryStatus.Empty
}
var sources = preferencesManager.dictionarySources.first() var sources = preferencesManager.dictionarySources.first()
if (downloadManager.downloadState.value !is DownloadState.Loading && if (pendingSourceUrls.isEmpty() &&
downloadManager.downloadState.value !is DownloadState.Loading &&
downloadManager.downloadState.value !is DownloadState.Extracting downloadManager.downloadState.value !is DownloadState.Extracting
) { ) {
val installedSources = installedSources(sources, actualDictionaries) val dictionaryFileNames = localDictionaryRepository.listDictionaryPayloadFileNames(
val installedSourceIds = installedSources.mapTo(mutableSetOf(), DictionarySource::id) preferencesManager.dictionaryPath
)
if (dictionaryFileNames != null) {
val installedSources = DictionarySourceFileMatcher.installedSourcesForFileNames(
sources,
dictionaryFileNames,
)
val installedSourceIds = installedSources
.mapTo(mutableSetOf(), DictionarySource::id)
val staleSourceIds = sources val staleSourceIds = sources
.filterNot { it.id in installedSourceIds } .filterNot { it.id in installedSourceIds }
.map(DictionarySource::id) .map(DictionarySource::id)
@@ -279,6 +308,12 @@ class SettingsViewModel(
sources = installedSources sources = installedSources
} }
} }
}
if (actualDictionaries.isEmpty()) {
return DictionaryStatus.Empty
}
val enabledSources = installedEnabledSources(sources, actualDictionaries) val enabledSources = installedEnabledSources(sources, actualDictionaries)
if (enabledSources.isEmpty()) { if (enabledSources.isEmpty()) {
@@ -326,26 +361,38 @@ class SettingsViewModel(
} }
private fun toggleDictionaryActive(dictionaryPath: String) { private fun toggleDictionaryActive(dictionaryPath: String) {
viewModelScope.launch {
localDictionaryRepository.toggleDictionaryActive(dictionaryPath) localDictionaryRepository.toggleDictionaryActive(dictionaryPath)
} }
}
private fun deleteDictionary(dictionary: Dictionary) { private fun deleteDictionary(dictionary: Dictionary) {
viewModelScope.launch { viewModelScope.launch {
localDictionaryRepository.deleteDictionary(dictionary) when (localDictionaryRepository.deleteDictionary(dictionary)) {
is OperationResult.Success -> {
manageDictionarySourcesUseCase.removeSourceForDictionary(dictionary) val remainingDictionaryFileNames =
localDictionaryRepository.listDictionaryPayloadFileNames(
val remainingDictionaries = localDictionaryRepository.dictionaries.first() preferencesManager.dictionaryPath
)
if (remainingDictionaries.isEmpty()) { manageDictionarySourcesUseCase.removeSourceForDictionary(
dictionaryStatus.value = DictionaryStatus.Empty dictionary = dictionary,
remainingDictionaryFileNames = remainingDictionaryFileNames,
)
dictionaryStatus.value = withContext(Dispatchers.IO) {
evaluateDictionaryStatus()
}
}
is OperationResult.Error -> {
effectChannel.send(
getApplication<Application>().getString(R.string.error_delete_dictionary)
)
}
} }
} }
} }
private fun addDictionarySources(urlTemplates: List<String>) { private fun addDictionarySources(urlTemplates: List<String>) {
viewModelScope.launch { viewModelScope.launch {
try {
val validUrls = mutableListOf<String>() val validUrls = mutableListOf<String>()
urlTemplates.forEach { urlTemplate -> urlTemplates.forEach { urlTemplate ->
@@ -354,23 +401,30 @@ class SettingsViewModel(
return@forEach return@forEach
} }
val normalizedUrl = DictionarySource.normalizeTemplate(trimmed)
if (!pendingSourceUrls.add(normalizedUrl)) {
return@forEach
}
try {
when (manageDictionarySourcesUseCase.addSource(trimmed)) { when (manageDictionarySourcesUseCase.addSource(trimmed)) {
is ManageDictionarySourcesUseCase.AddSourceResult.Success -> { is ManageDictionarySourcesUseCase.AddSourceResult.Success -> {
pendingSourceUrls.add(DictionarySource.normalizeTemplate(trimmed))
validUrls.add(trimmed) validUrls.add(trimmed)
} }
is ManageDictionarySourcesUseCase.AddSourceResult.ValidationFailed -> { is ManageDictionarySourcesUseCase.AddSourceResult.ValidationFailed -> {
pendingSourceUrls.remove(normalizedUrl)
// Validation failed, skip this source // Validation failed, skip this source
} }
} }
} catch (e: Exception) {
pendingSourceUrls.remove(normalizedUrl)
android.util.Log.e("SettingsViewModel", "Error adding dictionary source", e)
}
} }
if (validUrls.isNotEmpty()) { if (validUrls.isNotEmpty()) {
startDownloadForSources(validUrls) startDownloadForSources(validUrls)
} }
} catch (e: Exception) {
android.util.Log.e("SettingsViewModel", "Error adding dictionary sources", e)
}
} }
} }
@@ -383,6 +437,7 @@ class SettingsViewModel(
return@launch return@launch
} }
statusBeforeDownload = dictionaryStatus.value
dictionaryStatus.value = DictionaryStatus.Checking dictionaryStatus.value = DictionaryStatus.Checking
isDownloadInProgress = true isDownloadInProgress = true
@@ -406,11 +461,8 @@ class SettingsViewModel(
return@launch return@launch
} }
statusBeforeDownload = dictionaryStatus.value
dictionaryStatus.value = DictionaryStatus.Checking dictionaryStatus.value = DictionaryStatus.Checking
val status = withContext(Dispatchers.IO) { evaluateDictionaryStatus() }
dictionaryStatus.value = status
if (status !is DictionaryStatus.NeedsUpdate) return@launch
val installedSources = installedEnabledSources( val installedSources = installedEnabledSources(
preferencesManager.dictionarySources.first(), preferencesManager.dictionarySources.first(),
@@ -445,14 +497,8 @@ class SettingsViewModel(
private fun installedSources( private fun installedSources(
sources: List<DictionarySource>, sources: List<DictionarySource>,
dictionaries: List<Dictionary> dictionaries: List<Dictionary>
): List<DictionarySource> { ): List<DictionarySource> =
val installedFileNames = dictionaries.map { File(it.path).name } DictionarySourceFileMatcher.installedSources(sources, dictionaries)
return sources.filter { source ->
installedFileNames.any { fileName ->
DictionarySource.matchesDictionaryFile(source.urlTemplate, fileName)
}
}
}
private fun cancelDownload() { private fun cancelDownload() {
// Cancel the ViewModel-side scan job so no further status recomputes // Cancel the ViewModel-side scan job so no further status recomputes
@@ -481,13 +527,12 @@ class SettingsViewModel(
getApplication<Application>().stopService(stopIntent) getApplication<Application>().stopService(stopIntent)
} }
viewModelScope.launch {
val dictionaries = localDictionaryRepository.dictionaries.first()
dictionaryStatus.value = if (dictionaries.isNotEmpty()) {
DictionaryStatus.UpToDate
} else {
DictionaryStatus.Empty
} }
private fun refreshDictionaryStatus() {
viewModelScope.launch {
dictionaryStatus.value = DictionaryStatus.Checking
dictionaryStatus.value = withContext(Dispatchers.IO) { evaluateDictionaryStatus() }
} }
} }
@@ -499,10 +544,11 @@ class SettingsViewModel(
is OperationResult.Success -> { is OperationResult.Success -> {
val dictionaries = localDictionaryRepository.dictionaries.first() val dictionaries = localDictionaryRepository.dictionaries.first()
if (dictionaries.isEmpty()) { if (dictionaries.isEmpty()) {
dictionaryStatus.value = DictionaryStatus.Empty dictionaryStatus.value = withContext(Dispatchers.IO) {
evaluateDictionaryStatus()
}
isDownloadInProgress = false isDownloadInProgress = false
} else { } else if (isDownloadInProgress) {
if (isDownloadInProgress) {
dictionaryStatus.value = DictionaryStatus.UpToDate dictionaryStatus.value = DictionaryStatus.UpToDate
isDownloadInProgress = false isDownloadInProgress = false
} else { } else {
@@ -510,7 +556,6 @@ class SettingsViewModel(
val status = withContext(Dispatchers.IO) { evaluateDictionaryStatus() } val status = withContext(Dispatchers.IO) { evaluateDictionaryStatus() }
dictionaryStatus.value = status dictionaryStatus.value = status
} }
}
if (result.data > 0) { if (result.data > 0) {
viewModelScope.launch(Dispatchers.Default) { viewModelScope.launch(Dispatchers.Default) {
@@ -28,8 +28,10 @@ import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.rememberUpdatedState import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.runtime.setValue import androidx.compose.runtime.setValue
import androidx.compose.runtime.snapshotFlow
import androidx.compose.ui.Alignment import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clipToBounds
import androidx.compose.ui.draw.scale import androidx.compose.ui.draw.scale
import androidx.compose.ui.graphics.Color import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.graphicsLayer import androidx.compose.ui.graphics.graphicsLayer
@@ -81,7 +83,8 @@ fun DictionaryListItem(
dictionary: Dictionary, dictionary: Dictionary,
onToggle: () -> Unit, onToggle: () -> Unit,
onDelete: () -> Unit, onDelete: () -> Unit,
modifier: Modifier = Modifier modifier: Modifier = Modifier,
isDeleteBlocked: () -> Boolean = { false }
) { ) {
val haptic = LocalHapticFeedback.current val haptic = LocalHapticFeedback.current
val density = LocalDensity.current val density = LocalDensity.current
@@ -93,7 +96,10 @@ fun DictionaryListItem(
val offsetAnim = remember { Animatable(0f) } val offsetAnim = remember { Animatable(0f) }
var rawOffset by remember { mutableFloatStateOf(0f) } var rawOffset by remember { mutableFloatStateOf(0f) }
val isDeleteRevealed by remember { derivedStateOf { rawOffset <= -swipeThresholdPx } } val isDeleteBlockedCurrent by rememberUpdatedState(isDeleteBlocked)
val isDeleteRevealed by remember {
derivedStateOf { !isDeleteBlockedCurrent() && rawOffset <= -swipeThresholdPx }
}
LaunchedEffect(isDeleteRevealed) { LaunchedEffect(isDeleteRevealed) {
if (isDeleteRevealed) { if (isDeleteRevealed) {
@@ -101,6 +107,16 @@ fun DictionaryListItem(
} }
} }
LaunchedEffect(Unit) {
snapshotFlow { isDeleteBlockedCurrent() }
.collect { blocked ->
if (blocked) {
offsetAnim.snapTo(0f)
rawOffset = 0f
}
}
}
val currentOnDelete by rememberUpdatedState(onDelete) val currentOnDelete by rememberUpdatedState(onDelete)
val currentOnToggle by rememberUpdatedState(onToggle) val currentOnToggle by rememberUpdatedState(onToggle)
@@ -121,6 +137,8 @@ fun DictionaryListItem(
Box( Box(
modifier = modifier modifier = modifier
.fillMaxWidth() .fillMaxWidth()
.height(DictionaryItemHeight)
.clipToBounds()
.testTag("dictionary_item") .testTag("dictionary_item")
) { ) {
Row( Row(
@@ -132,12 +150,15 @@ fun DictionaryListItem(
.pointerInput(maxSwipePx, swipeThresholdPx) { .pointerInput(maxSwipePx, swipeThresholdPx) {
detectHorizontalDragGestures( detectHorizontalDragGestures(
onDragStart = { onDragStart = {
if (isDeleteBlockedCurrent()) return@detectHorizontalDragGestures
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove) haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
}, },
onDragEnd = { onDragEnd = {
if (isDeleteBlockedCurrent()) return@detectHorizontalDragGestures
val target = if (rawOffset <= -swipeThresholdPx) -maxSwipePx else 0f val target = if (rawOffset <= -swipeThresholdPx) -maxSwipePx else 0f
rawOffset = target rawOffset = target
scope.launch { scope.launch {
if (isDeleteBlockedCurrent()) return@launch
offsetAnim.animateTo( offsetAnim.animateTo(
target, target,
animationSpec = spring(stiffness = Spring.StiffnessMedium) animationSpec = spring(stiffness = Spring.StiffnessMedium)
@@ -145,9 +166,11 @@ fun DictionaryListItem(
} }
}, },
onDragCancel = { onDragCancel = {
if (isDeleteBlockedCurrent()) return@detectHorizontalDragGestures
val target = if (rawOffset <= -swipeThresholdPx) -maxSwipePx else 0f val target = if (rawOffset <= -swipeThresholdPx) -maxSwipePx else 0f
rawOffset = target rawOffset = target
scope.launch { scope.launch {
if (isDeleteBlockedCurrent()) return@launch
offsetAnim.animateTo( offsetAnim.animateTo(
target, target,
animationSpec = spring(stiffness = Spring.StiffnessMedium) animationSpec = spring(stiffness = Spring.StiffnessMedium)
@@ -155,9 +178,13 @@ fun DictionaryListItem(
} }
}, },
onHorizontalDrag = { _, dragAmount -> onHorizontalDrag = { _, dragAmount ->
if (isDeleteBlockedCurrent()) return@detectHorizontalDragGestures
val newOffset = (rawOffset + dragAmount).coerceIn(-maxSwipePx, 0f) val newOffset = (rawOffset + dragAmount).coerceIn(-maxSwipePx, 0f)
rawOffset = newOffset rawOffset = newOffset
scope.launch { offsetAnim.snapTo(newOffset) } scope.launch {
if (isDeleteBlockedCurrent()) return@launch
offsetAnim.snapTo(newOffset)
}
} }
) )
}, },
@@ -222,25 +249,11 @@ fun DictionaryListItem(
} }
if (isDeleteRevealed) { if (isDeleteRevealed) {
IconButton( Box(
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
currentOnDelete()
scope.launch { offsetAnim.snapTo(0f) }
rawOffset = 0f
},
modifier = Modifier modifier = Modifier
.padding(end = ItemHorizontalPadding) .padding(end = ItemHorizontalPadding)
.graphicsLayer { translationX = -offsetAnim.value }
.size(IconButtonSize) .size(IconButtonSize)
) {
Icon(
painter = painterResource(R.drawable.ic_delete),
contentDescription = stringResource(R.string.dictionary_delete),
tint = DeleteIconColor,
modifier = Modifier.size(IconSize)
) )
}
} else { } else {
Switch( Switch(
checked = dictionary.isActive, checked = dictionary.isActive,
@@ -257,5 +270,29 @@ fun DictionaryListItem(
) )
} }
} }
if (isDeleteRevealed) {
IconButton(
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
currentOnDelete()
scope.launch {
offsetAnim.snapTo(0f)
rawOffset = 0f
}
},
modifier = Modifier
.align(Alignment.CenterEnd)
.padding(end = ItemHorizontalPadding)
.size(IconButtonSize)
) {
Icon(
painter = painterResource(R.drawable.ic_delete),
contentDescription = stringResource(R.string.dictionary_delete),
tint = DeleteIconColor,
modifier = Modifier.size(IconSize)
)
}
}
} }
} }
@@ -11,8 +11,10 @@ import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.derivedStateOf import androidx.compose.runtime.derivedStateOf
import androidx.compose.runtime.getValue import androidx.compose.runtime.getValue
import androidx.compose.runtime.key
import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.ui.Alignment import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier import androidx.compose.ui.Modifier
import androidx.compose.ui.hapticfeedback.HapticFeedbackType import androidx.compose.ui.hapticfeedback.HapticFeedbackType
@@ -21,6 +23,7 @@ import androidx.compose.ui.platform.LocalFocusManager
import androidx.compose.ui.platform.LocalHapticFeedback import androidx.compose.ui.platform.LocalHapticFeedback
import androidx.compose.ui.res.pluralStringResource import androidx.compose.ui.res.pluralStringResource
import androidx.compose.ui.res.stringResource import androidx.compose.ui.res.stringResource
import androidx.compose.ui.semantics.invisibleToUser
import androidx.compose.ui.semantics.semantics import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import androidx.lifecycle.compose.collectAsStateWithLifecycle import androidx.lifecycle.compose.collectAsStateWithLifecycle
@@ -75,6 +78,13 @@ fun DictionaryManagement(
derivedStateOf { importState is ImportState.Importing || importState is ImportState.Extracting } derivedStateOf { importState is ImportState.Importing || importState is ImportState.Extracting }
} }
val hasEnabledSources by remember(sources) { derivedStateOf { sources.any { it.isEnabled } } } val hasEnabledSources by remember(sources) { derivedStateOf { sources.any { it.isEnabled } } }
val isBlockingMutations = isInProgress
// Invariant: the mutation-block flag is read only at state-read/gesture time (derivedStateOf,
// snapshotFlow, pointerInput callbacks), never at row composition time. A composition-time read
// (e.g. Modifier.alpha(if (isDeleteBlocked()) ...)) would invalidate every row on each flip.
val isMutationsBlockedState = rememberUpdatedState(isBlockingMutations)
val isDeleteBlocked = remember { { isMutationsBlockedState.value } }
SectionCard( SectionCard(
modifier = modifier modifier = modifier
@@ -125,13 +135,13 @@ fun DictionaryManagement(
// Disable accessibility on background content during import to improve performance. // Disable accessibility on background content during import to improve performance.
// The progress dialog remains accessible for cancellation. // The progress dialog remains accessible for cancellation.
val backgroundModifier = if (isInProgress) { val backgroundModifier = if (isBlockingMutations) {
Modifier.semantics(mergeDescendants = true) { } Modifier.semantics { invisibleToUser() }
} else { } else {
Modifier Modifier
} }
if (dictionaries.isEmpty() && !isInProgress) { if (dictionaries.isEmpty() && !isBlockingMutations) {
Column( Column(
modifier = backgroundModifier modifier = backgroundModifier
.fillMaxWidth() .fillMaxWidth()
@@ -153,7 +163,7 @@ fun DictionaryManagement(
) )
} }
} else { } else {
if (dictionaryStatus is DictionaryStatus.NeedsUpdate && !isInProgress && hasEnabledSources) { if (dictionaryStatus is DictionaryStatus.NeedsUpdate && !isBlockingMutations && hasEnabledSources) {
DictionaryUpdateCard( DictionaryUpdateCard(
onStartDownload = onStartDownload, onStartDownload = onStartDownload,
modifier = backgroundModifier modifier = backgroundModifier
@@ -166,10 +176,11 @@ fun DictionaryManagement(
onDeleteDictionary = onDeleteDictionary, onDeleteDictionary = onDeleteDictionary,
modifier = Modifier modifier = Modifier
.fillMaxWidth() .fillMaxWidth()
.then(backgroundModifier) .then(backgroundModifier),
isDeleteBlocked = isDeleteBlocked
) )
if (!isInProgress) { if (!isBlockingMutations) {
DictionaryActionButtons( DictionaryActionButtons(
onAddSource = { onAddSource = {
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove) haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
@@ -246,18 +257,22 @@ private fun DictionaryListSection(
dictionaries: List<Dictionary>, dictionaries: List<Dictionary>,
onToggleDictionary: (String) -> Unit, onToggleDictionary: (String) -> Unit,
onDeleteDictionary: (Dictionary) -> Unit, onDeleteDictionary: (Dictionary) -> Unit,
modifier: Modifier = Modifier modifier: Modifier = Modifier,
isDeleteBlocked: () -> Boolean = { false }
) { ) {
Column( Column(
modifier = modifier, modifier = modifier,
verticalArrangement = Arrangement.spacedBy(4.dp) verticalArrangement = Arrangement.spacedBy(4.dp)
) { ) {
dictionaries.forEach { dictionary -> dictionaries.forEach { dictionary ->
key(dictionary.path) {
DictionaryListItem( DictionaryListItem(
dictionary = dictionary, dictionary = dictionary,
onToggle = { onToggleDictionary(dictionary.path) }, onToggle = { onToggleDictionary(dictionary.path) },
onDelete = { onDeleteDictionary(dictionary) } onDelete = { onDeleteDictionary(dictionary) },
isDeleteBlocked = isDeleteBlocked
) )
} }
} }
}
} }
@@ -1,8 +1,8 @@
package com.example.research.ui.settings.components package com.example.research.ui.settings.components
import androidx.compose.animation.AnimatedVisibility import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.core.FastOutSlowInEasing import androidx.compose.animation.core.Animatable
import androidx.compose.animation.core.animateFloatAsState import androidx.compose.animation.core.LinearEasing
import androidx.compose.animation.core.tween import androidx.compose.animation.core.tween
import androidx.compose.animation.expandVertically import androidx.compose.animation.expandVertically
import androidx.compose.animation.fadeIn import androidx.compose.animation.fadeIn
@@ -19,6 +19,10 @@ import androidx.compose.ui.res.stringResource
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import com.example.research.R import com.example.research.R
private const val MILLIS_PER_PROGRESS_UNIT = 10_000
private const val MIN_PROGRESS_MOTION_MS = 50
private const val MAX_PROGRESS_MOTION_MS = 650
@Composable @Composable
fun DictionaryProgressSection( fun DictionaryProgressSection(
visible: Boolean, visible: Boolean,
@@ -41,6 +45,11 @@ fun DictionaryProgressSection(
var displayedPercent by remember { mutableIntStateOf(percent) } var displayedPercent by remember { mutableIntStateOf(percent) }
var displayedTestTag by remember { mutableStateOf(testTag) } var displayedTestTag by remember { mutableStateOf(testTag) }
LaunchedEffect(visible) {
if (visible) {
monotonicTargetProgress = targetProgress
}
}
LaunchedEffect(visible, targetProgress) { LaunchedEffect(visible, targetProgress) {
if (visible) { if (visible) {
monotonicTargetProgress = maxOf(monotonicTargetProgress, targetProgress) monotonicTargetProgress = maxOf(monotonicTargetProgress, targetProgress)
@@ -53,11 +62,21 @@ fun DictionaryProgressSection(
displayedTestTag = testTag displayedTestTag = testTag
} }
} }
val animatedProgress by animateFloatAsState( val animatedState = remember { Animatable(0f) }
LaunchedEffect(monotonicTargetProgress) {
val delta = monotonicTargetProgress - animatedState.value
if (delta > 0f) {
animatedState.animateTo(
targetValue = monotonicTargetProgress, targetValue = monotonicTargetProgress,
animationSpec = tween(durationMillis = 650, easing = FastOutSlowInEasing), animationSpec = tween(
label = "dictionary-progress" durationMillis = (delta * MILLIS_PER_PROGRESS_UNIT).toInt()
.coerceIn(MIN_PROGRESS_MOTION_MS, MAX_PROGRESS_MOTION_MS),
easing = LinearEasing,
),
) )
}
}
val animatedProgress = animatedState.value
Column(modifier = Modifier.testTag(displayedTestTag)) { Column(modifier = Modifier.testTag(displayedTestTag)) {
Row( Row(
+4 -2
View File
@@ -7,11 +7,11 @@
<string name="button_cancel_download">Отмена</string> <string name="button_cancel_download">Отмена</string>
<string name="action_ok">ОК</string> <string name="action_ok">ОК</string>
<string name="download_error">Ошибка загрузки словарей</string> <string name="download_error">Ошибка загрузки словарей</string>
<string name="download_network_failed">Нет связи с сервером словарей. Проверьте сеть и попробуйте снова</string>
<string name="language">Язык</string> <string name="language">Язык</string>
<string name="language_english">Английский</string> <string name="language_english">Английский</string>
<string name="language_russian">Русский</string> <string name="language_russian">Русский</string>
<string name="language_system">Системный</string> <string name="language_system">Системный</string>
<string name="indexing_label">Индексация словарей</string>
<string name="clear_search">Очистить поиск</string> <string name="clear_search">Очистить поиск</string>
<string name="article_no_selected">Статья не выбрана</string> <string name="article_no_selected">Статья не выбрана</string>
<string name="article_return_to_search">Вернуться к поиску</string> <string name="article_return_to_search">Вернуться к поиску</string>
@@ -67,8 +67,9 @@
<string name="dictionary_management_title">Словари</string> <string name="dictionary_management_title">Словари</string>
<string name="import_dictionary_button">Выбрать файлы</string> <string name="import_dictionary_button">Выбрать файлы</string>
<string name="import_error">Не удалось импортировать словарь: %1$s</string> <string name="import_error">Не удалось импортировать словарь: %1$s</string>
<string name="import_file_exists">Файл уже существует: %1$s</string> <string name="import_file_exists">Словарь уже добавлен: %1$s</string>
<string name="import_invalid_file_name">Недопустимое имя файла: %1$s</string> <string name="import_invalid_file_name">Недопустимое имя файла: %1$s</string>
<string name="import_nothing_imported">В выбранном нет поддерживаемых файлов словарей</string>
<string name="dictionary_source_url_hint">URL</string> <string name="dictionary_source_url_hint">URL</string>
<string name="dictionary_source_add_button">Добавить источник</string> <string name="dictionary_source_add_button">Добавить источник</string>
<string name="dictionary_source_duplicate">Этот URL уже существует</string> <string name="dictionary_source_duplicate">Этот URL уже существует</string>
@@ -95,6 +96,7 @@
</plurals> </plurals>
<string name="dictionary_not_indexed">• Не индексирован</string> <string name="dictionary_not_indexed">• Не индексирован</string>
<string name="dictionary_delete">Удалить словарь</string> <string name="dictionary_delete">Удалить словарь</string>
<string name="error_delete_dictionary">Не удалось удалить словарь</string>
<string name="dictionaries_tap_to_update">Нажмите для обновления</string> <string name="dictionaries_tap_to_update">Нажмите для обновления</string>
<string name="notification_import_title">Импорт словарей</string> <string name="notification_import_title">Импорт словарей</string>
<string name="notification_import_success_title">Импорт завершён</string> <string name="notification_import_success_title">Импорт завершён</string>
+4 -2
View File
@@ -7,11 +7,11 @@
<string name="button_cancel_download">Cancel</string> <string name="button_cancel_download">Cancel</string>
<string name="action_ok">OK</string> <string name="action_ok">OK</string>
<string name="download_error">Dictionary download failed</string> <string name="download_error">Dictionary download failed</string>
<string name="download_network_failed">No connection to the dictionary server. Check your network and try again</string>
<string name="language">Language</string> <string name="language">Language</string>
<string name="language_english">English</string> <string name="language_english">English</string>
<string name="language_russian">Russian</string> <string name="language_russian">Russian</string>
<string name="language_system">System</string> <string name="language_system">System</string>
<string name="indexing_label">Indexing dictionaries</string>
<string name="clear_search">Clear search</string> <string name="clear_search">Clear search</string>
<string name="article_no_selected">No article selected</string> <string name="article_no_selected">No article selected</string>
<string name="article_return_to_search">Return to search</string> <string name="article_return_to_search">Return to search</string>
@@ -65,8 +65,9 @@
<string name="dictionary_management_title">Dictionaries</string> <string name="dictionary_management_title">Dictionaries</string>
<string name="import_dictionary_button">Select files</string> <string name="import_dictionary_button">Select files</string>
<string name="import_error">Failed to import dictionary: %1$s</string> <string name="import_error">Failed to import dictionary: %1$s</string>
<string name="import_file_exists">File already exists: %1$s</string> <string name="import_file_exists">Dictionary already added: %1$s</string>
<string name="import_invalid_file_name">Invalid file name: %1$s</string> <string name="import_invalid_file_name">Invalid file name: %1$s</string>
<string name="import_nothing_imported">No supported dictionary files were found in the selection</string>
<string name="dictionary_source_url_hint">URL</string> <string name="dictionary_source_url_hint">URL</string>
<string name="dictionary_source_add_button">Add source</string> <string name="dictionary_source_add_button">Add source</string>
<string name="dictionary_source_duplicate">This URL already exists</string> <string name="dictionary_source_duplicate">This URL already exists</string>
@@ -91,6 +92,7 @@
</plurals> </plurals>
<string name="dictionary_not_indexed">• Not indexed</string> <string name="dictionary_not_indexed">• Not indexed</string>
<string name="dictionary_delete">Delete dictionary</string> <string name="dictionary_delete">Delete dictionary</string>
<string name="error_delete_dictionary">Failed to delete dictionary</string>
<string name="dictionaries_tap_to_update">Tap to update all dictionaries</string> <string name="dictionaries_tap_to_update">Tap to update all dictionaries</string>
<string name="notification_import_title">Importing dictionaries</string> <string name="notification_import_title">Importing dictionaries</string>
<string name="notification_import_success_title">Import completed</string> <string name="notification_import_success_title">Import completed</string>
@@ -0,0 +1,3 @@
- Fixed the dictionary update card after starting or cancelling a download
- Fixed download progress staying at 0% for too long
- Rebuilt the search field on Material 3 and aligned dictionary title typography
@@ -0,0 +1,4 @@
- Fixed a crash and lost dictionary state around search results
- Added diacritic-insensitive search, so accented and plain spellings match each other
- Fixed indexing failures on dictionaries with dense non-Latin text
- Search results now keep loading as you scroll instead of stopping early
@@ -0,0 +1 @@
- Fixed the dictionary update card taking too long to reappear after cancelling a download
@@ -0,0 +1,2 @@
- Dictionaries can no longer be deleted while an update is running, which used to lose their update source
- Fixed the dictionary icon sliding outside the row while swiping to delete
@@ -0,0 +1 @@
- Fixed matching source URLs to installed dictionary files, so deleted dictionaries can be downloaded again from the same URL
@@ -0,0 +1,3 @@
- The dictionary update card now returns after a failed download
- Fixed headwords and articles with escaped characters being displayed incorrectly
- Fixed slow rendering of articles containing many bracket characters
@@ -9,6 +9,6 @@ Features:
- Automatic indexing with binary index files for instant search - Automatic indexing with binary index files for instant search
- Multi-charset support — UTF-8, UTF-16 LE/BE auto-detection - Multi-charset support — UTF-8, UTF-16 LE/BE auto-detection
- Adaptive three-pane layout for tablets and large screens - Adaptive three-pane layout for tablets and large screens
- Light, dark, and system-follow themes - Material 3 UI with dynamic color, plus light, dark, and system-follow themes
- English and Russian localization - English and Russian localization
- No tracking, no ads, fully open source - No tracking, no ads, fully open source
Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

@@ -0,0 +1,3 @@
- Исправлена карточка обновления словаря после запуска и отмены загрузки
- Исправлено зависание индикатора прогресса загрузки на 0%
- Строка поиска переведена на Material 3; выровнена типографика заголовков словаря
@@ -0,0 +1,4 @@
- Исправлен сбой и потеря состояния словаря в результатах поиска
- Добавлен поиск без учёта диакритических знаков — совпадают варианты с акцентами и без
- Исправлена ошибка индексации словарей с плотным нелатинским текстом
- Результаты поиска теперь полностью подгружаются при прокрутке, а не обрываются на первых совпадениях
@@ -0,0 +1 @@
- Исправлена лишняя задержка перед повторным появлением карточки обновления словаря после отмены загрузки
@@ -0,0 +1,2 @@
- Словари больше нельзя удалить во время обновления — раньше при этом терялся источник обновления
- Исправлен выход иконки словаря за границы строки при свайпе
@@ -0,0 +1 @@
- Исправлено сопоставление URL-источников с установленными файлами — удалённый словарь теперь можно повторно скачать по тому же URL
@@ -0,0 +1,3 @@
- Карточка обновления словарей теперь появляется снова после неудачной загрузки
- Исправлено отображение заголовков и статей с экранированными символами
- Исправлено медленное отображение статей с большим количеством скобок
@@ -9,6 +9,6 @@ ReSearch — быстрое приложение для чтения слова
- Автоматическая индексация с бинарными индексными файлами для мгновенного поиска - Автоматическая индексация с бинарными индексными файлами для мгновенного поиска
- Поддержка нескольких кодировок — автоопределение UTF-8, UTF-16 LE/BE - Поддержка нескольких кодировок — автоопределение UTF-8, UTF-16 LE/BE
- Адаптивный трёхпанельный интерфейс для планшетов и больших экранов - Адаптивный трёхпанельный интерфейс для планшетов и больших экранов
- Светлая, тёмная и системная темы - Material 3 с динамическими цветами, светлая, тёмная и системная темы
- Локализация на английский и русский языки - Локализация на английский и русский языки
- Без трекеров, без рекламы, полностью открытый исходный код - Без трекеров, без рекламы, полностью открытый исходный код
Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

+6 -6
View File
@@ -1,15 +1,15 @@
[versions] [versions]
aboutlibraries = "15.0.3" aboutlibraries = "15.2.0"
activity_compose = "1.13.0" activity_compose = "1.13.0"
agp = "9.2.1" agp = "9.3.1"
compose_bom = "2026.06.01" compose_bom = "2026.08.00"
core = "1.19.0" core = "1.19.0"
datastore_preferences = "1.2.1" datastore_preferences = "1.2.1"
documentfile = "1.1.0" documentfile = "1.1.0"
kotlin = "2.4.0" kotlin = "2.4.10"
lifecycle_runtime_ktx = "2.11.0" lifecycle_runtime_ktx = "2.11.0"
okhttp = "5.4.0" okhttp = "5.5.0"
paging = "3.5.0" paging = "3.5.1"
profileinstaller = "1.4.1" profileinstaller = "1.4.1"
kotlinx_coroutines = "1.11.0" kotlinx_coroutines = "1.11.0"
kotlinx_serialization = "1.11.0" kotlinx_serialization = "1.11.0"
+1 -1
View File
@@ -1,6 +1,6 @@
distributionBase=GRADLE_USER_HOME distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.0-bin.zip distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.1-bin.zip
networkTimeout=10000 networkTimeout=10000
retries=0 retries=0
retryBackOffMs=500 retryBackOffMs=500