30 Commits
Author SHA1 Message Date
OneWay 380e4ca916 fix: latch terminal notification state and post pipeline progress on one thread
A late ImportState Success snapshot collected after the success
notification overwrote it with an ongoing progress record. Post
progress on Main behind a volatile terminal latch; notifications go
through NotificationPort, the pipeline lives in the coordinator, and
the service finish is injected.
2026-09-06 15:32:35 +08:00
OneWay 47b1197a92 fix: defer import errors until the copy loop finishes
Errors and skips are collected while the copy loop runs and reported
once it ends. The final import state is exhaustive - success, already
added, invalid name, or nothing supported found - so every import run
reaches a terminal state and the pipeline notification always resolves
instead of lingering as an indeterminate progress bar.
2026-09-05 21:55:01 +08:00
OneWay c1563b8172 fix: tie back arrow to pipeline idle state
Computed from operation states in SettingsViewModel instead of a fixed
delay; cancel and error release the arrow immediately.
2026-09-05 05:55:56 +08:00
OneWay 4a00d8adff fix: key search result rows by dictionary, offset and word
Paging result rows had no keys and were matched by position: appended
pages and query diffs could not reuse row state. Keys derived from
dictionaryPath+offset+word are stable per row, and the paging source
collapses duplicate entries for one dictionary card: case and diacritic
alias variants of a headword share the normalized word, so entries are
deduplicated by dictionaryPath+word+offset before emitting.
2026-09-05 03:00:44 +08:00
OneWay 944d2e6634 perf(about): parse aboutlibraries.json off the main thread
Bump AboutLibraries plugin to 15.0.4 -> 15.2.0; parse
aboutlibraries.json on Dispatchers.IO via produceState (header renders
immediately, rows fill in); drop animateContentSize from rows.
2026-09-05 00:37:41 +08:00
OneWay d7de09a152 fix: open Settings immediately on cold start when no dictionaries
Seed and synchronously read hadNoDictionaries before setContent so
Settings shows from the first frame until the startup scan confirms.
2026-09-04 19:09:19 +08:00
OneWay e21a134d65 fix: hide dictionary rows from accessibility while mutations blocked
invisibleToUser() removes blocked content from the accessibility tree;
mergeDescendants only regrouped traversal order.
2026-09-04 18:04:32 +08:00
OneWay b9baf61264 fix: re-check mutation-block inside gesture coroutines
Stale drag-tick coroutines launched before the flip no-op at execution
time instead of moving the row after the reset.
2026-09-04 14:59:59 +08:00
OneWay 7b63ee4377 refactor: unify swipe reset to snap before clearing offset
Snap the Animatable first, clear rawOffset after - no frame with the
row still translated while the delete affordance is hidden.
2026-09-04 13:28:27 +08:00
OneWay d892427a3f fix: key dictionary list items by path
Swipe state follows its row instead of migrating to a neighbor on
delete or reorder.
2026-09-04 02:05:59 +08:00
OneWay 628813c3f6 refactor: isolate dictionary rows from mutation-block recomposition
Delete-block flag becomes a () -> Boolean provider read at state/gesture
time, so flipping it no longer recomposes the whole Dictionaries block.
2026-09-04 01:53:06 +08:00
OneWay 4f202f8c10 refactor: replace deprecated currentWindowAdaptiveInfo with V2 2026-09-02 16:18:39 +08:00
OneWay 44ebe68f75 refactor: expose state flows via asStateFlow
Drops the explicit getters and an unused import, and lets an emptyList
type argument be inferred.
2026-09-02 12:55:48 +08:00
OneWay 5de466792a chore: remove unused index comparison and decode helpers
IndexSearcher.compareEntryGroups / IndexComparisonSummary /
IndexComparisonException / readEntryOrNull and AboutLibrariesData.decode
have no call sites left in the app.
2026-09-02 11:30:51 +08:00
OneWay b6e6bd328c fix: reset dictionary progress bar between indexing operations
monotonicTargetProgress is held in remember and only ever grows via
maxOf, so when a new indexing starts after a previous one reached a
higher value the bar is left at the old level while the percent label
starts at 0. Snap it to the current target when the section becomes
visible so the bar follows the new operation from its first frame.
2026-09-02 10:56:32 +08:00
OneWay 5132ae66b9 fix: cancel stale progress notification on process start
Notifications belong to the package and outlive the process: after the
system kills the foreground service mid-download the frozen progress
notification stays in the shade forever. A fresh process means no
service is running, so any leftover notification is stale.
2026-09-02 08:37:43 +08:00
OneWay 5151665216 perf: mark Compose state types as @Immutable
Compose compiler reports showed DownloadState's sealed root as Uncertain
and DownloadProgressState/SettingsUiState as runtime stability, so
skippability checks were deferred to runtime. The annotations make them
compile-time stable and turn any future mutable field into an explicit
contract break.
2026-09-01 17:28:17 +08:00
OneWay 96461bd7cf fix: probe source availability before showing download progress
Probe the source URLs up front so the download flow only enters the
progress state when something is actually downloadable, and report
Idle instead of a fake Success when nothing is: the early return
skipped the terminal-state delay and reset, leaving a phantom 100%
Success (and a stuck "indexing" bar plus a success notification) for
downloads that never happened.
2026-09-01 11:45:41 +08:00
OneWay 98ef99ab53 fix: probe source availability before showing download progress
Instant server failures (404/HTML error page) no longer flash a 0%
progress bar; the check mirrors the downloader's previous-day fallback
and tolerates transient network errors, which still go through the
downloader's retry path.
2026-09-05 10:03:54 +08:00
OneWay 43823e3bfa Fix: clear search results instantly on query clear
Skipped the 300ms debounce for blank queries so the results list clears immediately; typed queries keep the debounce.
2026-08-30 03:36:27 +08:00
OneWay bab03b6c73 fix: retry dictionary downloads on network-switch errors
Downloads killed by WiFi/mobile transitions failed with the generic
"Dictionary download failed" message because UnknownHostException,
ConnectException, unexpected end of stream, and closed-socket errors
were not classified as network errors and were never retried.
Classify them by exception type and message so the retry loop
engages and exhaustion reports the network-specific message.

Bump OkHttp 5.4.0 to 5.5.0: a throwing system proxy selector now
falls back to a direct connection instead of failing the call,
complementing the download resilience fix.
2026-08-30 00:39:04 +08:00
OneWay d4e6302274 docs: update README and Fastlane descriptions 2026-08-25 23:59:58 +08:00
OneWay bc3fa85b9d Release 1.5.0
- The dictionary update card now returns after a failed download
- Fixed headwords and articles with escaped characters being displayed incorrectly
- Fixed slow rendering of articles containing many bracket characters
2026-08-18 00:16:27 +03:00
OneWay 457bbe43b6 Unescape any escaped character in article text
Apply the general DSL rule \X -> X instead of unescaping only brackets
and parentheses, so sequences like \! no longer show the backslash.
2026-08-18 00:15:53 +03:00
OneWay f751ac0d36 Harden DSL parsing against escapes and malformed tags
Fixed escape-aware headword parsing for \{, \}, \[ and trailing backslash.
Fixed O(n^2) tag scan and nested bracket handling in article rendering.
2026-08-16 11:13:46 +03:00
OneWay 910bc8313c Release 1.4.0
- Fixed matching source URLs to installed dictionary files, so deleted dictionaries can be downloaded again from the same URL
2026-08-06 14:26:03 +03:00
OneWay 9d18d0c927 Release 1.3.0
- Blocked swipe-to-delete on dictionary rows while a download, extract,
  import or indexing pass is running. Deleting mid-update removed the
  dictionary's source from preferences while the pipeline kept its own
  snapshot, so the post-update rescan re-created the dictionary with its
  update source permanently lost.
- Fixed the dictionary icon leaving the row while swiping to delete: the
  item now clips to its bounds and the delete button is a static overlay
  instead of a counter-translated child of the sliding row.
2026-08-02 11:09:47 +03:00
OneWay 69bbb97b2a Release 1.2.1
- Fixed the dictionary update card taking too long to reappear after
  cancelling a download (DownloadManager).
- Bump agp 9.3.0 -> 9.3.1; migrate the release build type from
  isMinifyEnabled/isShrinkResources/proguardFiles to
  optimization { enable = true } with keep rules moved to
  app/src/release/keepRules/proguard-rules.keep.
2026-07-31 10:47:29 +03:00
OneWay c8adf537d4 Add app icon to fastlane metadata 2026-07-30 19:46:12 +03:00
OneWay 84612b26f2 Release 1.2.0
- Fixed a crash on broad queries (e.g. a single letter across several
  dictionaries): the results list keyed items by dictionary
  path/offset/word, and duplicate keys are illegal in a Compose list;
  drop the explicit key, let Paging identify items by position instead
  (SearchResultsList).
- Fixed a disabled dictionary reactivating itself on the next app
  launch: the active flag is now persisted through
  PreferencesManager/DataStore and reapplied whenever the dictionary
  list is rebuilt (LocalDictionaryRepository).
- Search results no longer stop at the first ~100-200 matches: the
  ranked head page hands back a cursor per unfinished prefix range,
  and IndexEntryPagingSource pages the rest straight from the index,
  merged alphabetically across dictionaries, until every match is
  surfaced (IndexSearcher, IndexEntryPagingSource).
- Added diacritic-insensitive search: folded index entries are merged
  into the primary index as aliases, so accented and plain spellings
  find each other (DslIndexer, IndexSearcher).
- Fixed indexing failures on dictionaries with dense non-Latin text: a
  truncated multi-byte sequence at a sample boundary was misread as
  invalid UTF-8 and the whole file fell back to UTF-16LE
  (DslCharsetDetector).
2026-07-22 16:29:16 +03:00
67 changed files with 1710 additions and 852 deletions
+6 -6
View File
@@ -29,7 +29,7 @@ The app stores dictionaries as DictZip files and builds compact binary indexes f
- Random-access article loading from DictZip dictionaries.
- Background processing with progress shown in Settings and notifications.
- Dictionary management with enable, disable, update, and delete actions.
- Material 3 interface with light, dark, and system themes.
- Material 3 interface with dynamic color, light, dark, and system themes.
- English and Russian localization.
- About screen with open source license information.
@@ -94,16 +94,16 @@ The main search screen remains usable as long as at least one indexed dictionary
## Tech Stack
- Kotlin 2.4.0
- Jetpack Compose (BOM 2026.06.01)
- Kotlin 2.4.10
- Jetpack Compose (BOM 2026.08.00)
- Material 3
- Coroutines and Flow 1.11.0
- DataStore Preferences 1.2.1
- Paging 3.5.0
- Paging 3.5.1
- OkHttp 5.4.0
- kotlinx.serialization 1.11.0
- AboutLibraries 15.0.3 metadata generation
- Android Gradle Plugin 9.2.1
- AboutLibraries 15.0.4 metadata generation
- Android Gradle Plugin 9.3.1
## Build
+5 -14
View File
@@ -92,14 +92,8 @@ android {
applicationId = "com.example.research"
minSdk = project.property("minSdk").toString().toInt()
targetSdk = project.property("targetSdk").toString().toInt()
versionCode = 3
versionName = "1.1.1"
val isFdroid = project.hasProperty("fdroid")
if (isFdroid) {
applicationIdSuffix = ".fdroid"
versionNameSuffix = "-fdroid"
}
versionCode = 8
versionName = "1.5.0"
vectorDrawables {
useSupportLibrary = true
@@ -120,12 +114,9 @@ android {
abiFilters += "arm64-v8a"
}
isMinifyEnabled = true
isShrinkResources = true
proguardFiles(
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro"
)
optimization {
enable = true
}
}
}
@@ -1,5 +1,8 @@
package com.example.research
import androidx.compose.runtime.Immutable
@Immutable
sealed interface DictionaryStatus {
data object Unknown : DictionaryStatus
data object Checking : DictionaryStatus
@@ -83,6 +83,7 @@ class MainActivity : ComponentActivity() {
val app = application as ReSearchApplication
val initialTheme = loadInitialTheme(app)
val initialLanguage = loadInitialLanguage(app)
val initialHadNoDictionaries = loadInitialHadNoDictionaries(app)
handleIntent(intent)
setContent {
@@ -104,7 +105,8 @@ class MainActivity : ComponentActivity() {
) {
AppNavigation(
searchViewModel = searchViewModel,
settingsViewModel = settingsViewModel
settingsViewModel = settingsViewModel,
seedNoDictionaries = initialHadNoDictionaries
)
}
}
@@ -165,6 +167,16 @@ class MainActivity : ComponentActivity() {
}
}
private fun loadInitialHadNoDictionaries(app: ReSearchApplication): Boolean {
return try {
runBlocking {
app.preferencesManager.hadNoDictionaries.first()
}
} catch (e: Exception) {
true
}
}
override fun onNewIntent(intent: Intent) {
super.onNewIntent(intent)
handleIntent(intent)
@@ -2,6 +2,7 @@ package com.example.research
import android.app.Application
import com.example.research.common.progress.DictionaryProgressStateHolder
import com.example.research.common.util.NotificationHelper
import com.example.research.data.local.preferences.PreferencesManager
import com.example.research.data.repository.LocalDictionaryRepository
import com.example.research.feature.download.DownloadManager
@@ -36,16 +37,23 @@ class ReSearchApplication : Application() {
private set
lateinit var dictionaryProgressStateHolder: DictionaryProgressStateHolder
private set
lateinit var dictionaryProgressPresenter: com.example.research.common.progress.DictionaryProgressPresenter
private set
lateinit var dictionaryPipelineCoordinator: com.example.research.common.progress.DictionaryPipelineCoordinator
private set
private val applicationScope = CoroutineScope(SupervisorJob() + Dispatchers.Default)
override fun onCreate() {
super.onCreate()
val notificationHelper = NotificationHelper(this)
notificationHelper.cancelProgressNotification()
preferencesManager = PreferencesManager(this)
applicationScope.launch(Dispatchers.IO) {
preferencesManager.sanitizeDictionarySources()
}
localDictionaryRepository = LocalDictionaryRepository(this)
localDictionaryRepository = LocalDictionaryRepository(this, preferencesManager = preferencesManager)
okHttpClient = OkHttpClient.Builder()
.protocols(listOf(Protocol.HTTP_1_1))
@@ -62,7 +70,8 @@ class ReSearchApplication : Application() {
downloadManager = DownloadManager(
dictionaryRepository = downloadDictionaryRepository,
unknownErrorMessage = getString(R.string.unknown_error)
unknownErrorMessage = getString(R.string.unknown_error),
sourceUnavailableMessage = getString(R.string.download_file_not_found)
)
dictionaryImportManager = DictionaryImportManager(
@@ -76,6 +85,40 @@ class ReSearchApplication : Application() {
localDictionaryRepository = localDictionaryRepository,
)
dictionaryProgressPresenter = com.example.research.common.progress.DictionaryProgressPresenter(
scope = applicationScope,
progressSnapshot = dictionaryProgressStateHolder.progressSnapshot,
notifications = notificationHelper,
)
dictionaryPipelineCoordinator = com.example.research.common.progress.DictionaryPipelineCoordinator(
scope = applicationScope,
steps = com.example.research.common.progress.DefaultReindexingSteps(
localDictionaryRepository = localDictionaryRepository,
downloadDictionaryRepository = downloadDictionaryRepository,
),
importOperations = dictionaryImportManager,
ui = dictionaryProgressPresenter,
dictionaryPath = { preferencesManager.dictionaryPath },
finishService = {
val intent = android.content.Intent(
this,
com.example.research.feature.download.service.DictionaryForegroundService::class.java
).apply {
action = com.example.research.feature.download.service.DictionaryForegroundService.ACTION_FINISH
}
try {
startService(intent)
} catch (_: Exception) {
}
},
)
dictionaryImportManager.onFlowStarted = { dictionaryProgressPresenter.beginFlow() }
dictionaryImportManager.onTerminal = dictionaryPipelineCoordinator::onImportTerminal
downloadManager.onFlowStarted = { dictionaryProgressPresenter.beginFlow() }
downloadManager.onDownloadSuccess = dictionaryPipelineCoordinator::onDownloadSuccess
downloadManager.onTerminal = dictionaryPipelineCoordinator::onDownloadTerminal
}
override fun onTrimMemory(level: Int) {
@@ -0,0 +1,131 @@
package com.example.research.common.progress
import com.example.research.core.util.OperationResult
import com.example.research.feature.download.model.DownloadState
import com.example.research.ui.settings.ImportState
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
import java.io.File
import kotlin.time.Duration.Companion.seconds
class DictionaryPipelineCoordinator(
private val scope: CoroutineScope,
private val steps: ReindexingSteps,
private val importOperations: ImportFlowOperations,
private val ui: PipelineTerminalUi,
private val dictionaryPath: () -> String,
private val finishService: () -> Unit,
private val launchDispatcher: CoroutineDispatcher = Dispatchers.Main,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO,
) {
private var pipelineJob: Job? = null
fun onImportTerminal(state: ImportState) {
when (state) {
is ImportState.Success -> runPipeline(isImport = true)
is ImportState.Error -> {
ui.showError()
finishService()
}
else -> {}
}
}
fun onDownloadSuccess() = runPipeline(isImport = false)
fun onDownloadTerminal(state: DownloadState) {
when (state) {
is DownloadState.Error -> {
ui.showError()
finishService()
}
is DownloadState.Cancelled -> {
ui.cancelProgress()
finishService()
}
else -> {}
}
}
fun cancelPipeline() {
pipelineJob?.cancel()
pipelineJob = null
}
private fun runPipeline(isImport: Boolean) {
pipelineJob?.cancel()
pipelineJob = scope.launch(launchDispatcher) {
val dir = if (isImport) File(dictionaryPath()) else null
val filesBeforeReindex = dir?.listFiles()?.map { it.name }?.toSet() ?: emptySet()
try {
triggerReindexing(isImport)
if (isImport) {
importOperations.getAndClearImportedFiles()
importOperations.clearImportState()
ui.showImportSuccess()
} else {
ui.showDownloadSuccess()
}
} catch (e: CancellationException) {
if (isImport) {
val filesToCleanup = importOperations.getAndClearImportedFiles()
filesToCleanup.forEach { file ->
try {
if (file.exists()) file.delete()
} catch (_: Exception) {
}
}
cleanupNewFiles(dir, filesBeforeReindex)
}
throw e
} catch (e: Exception) {
ui.showError()
} finally {
finishService()
}
}
}
private suspend fun triggerReindexing(isImport: Boolean) = withContext(ioDispatcher) {
steps.waitForIndexingCompletion()
delay(1.seconds)
val path = dictionaryPath()
if (isImport) {
importOperations.updateExtractionProgress(0f)
}
steps.extractArchives(onProgress = { progress ->
if (isImport) {
importOperations.updateExtractionProgress(progress)
}
})
val result = steps.scanDirectory(path)
if (result is OperationResult.Success && result.data > 0) {
steps.warmupIndexes()
steps.performAllCleanup()
}
if (isImport) {
importOperations.markImportPipelineSuccess()
}
}
private fun cleanupNewFiles(dir: File?, filesBeforeSnapshot: Set<String>) {
try {
dir?.listFiles()?.forEach { file ->
if (file.name !in filesBeforeSnapshot) {
try {
file.delete()
} catch (_: Exception) {
}
}
}
} catch (_: Exception) {
}
}
}
@@ -0,0 +1,56 @@
package com.example.research.common.progress
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.launch
class DictionaryProgressPresenter(
private val scope: CoroutineScope,
private val progressSnapshot: StateFlow<DictionaryProgressModel.Snapshot?>,
private val notifications: NotificationPort,
mainDispatcher: CoroutineDispatcher = Dispatchers.Main,
) : PipelineTerminalUi {
@Volatile
private var terminalActive = false
init {
scope.launch(mainDispatcher) {
progressSnapshot.collect { snapshot ->
when {
snapshot == null -> {
if (!terminalActive && notifications.hasActiveProgressNotification()) {
notifications.cancelProgressNotification()
}
}
!terminalActive -> notifications.showProgress(snapshot.percent, snapshot.titleRes)
}
}
}
}
fun beginFlow() {
terminalActive = false
}
override fun showImportSuccess() {
notifications.showImportSuccess()
terminalActive = true
}
override fun showDownloadSuccess() {
notifications.showDownloadSuccess()
terminalActive = true
}
override fun showError() {
notifications.showError()
terminalActive = true
}
override fun cancelProgress() {
terminalActive = false
notifications.cancelProgressNotification()
}
}
@@ -0,0 +1,10 @@
package com.example.research.common.progress
interface NotificationPort {
fun showProgress(percent: Int, titleRes: Int)
fun showImportSuccess()
fun showDownloadSuccess()
fun showError()
fun cancelProgressNotification()
fun hasActiveProgressNotification(): Boolean
}
@@ -0,0 +1,65 @@
package com.example.research.common.progress
import com.example.research.core.util.OperationResult
import com.example.research.data.repository.LocalDictionaryRepository
import com.example.research.feature.download.repository.DictionaryRepository
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.first
import java.io.File
import kotlin.time.Duration.Companion.milliseconds
interface PipelineTerminalUi {
fun showImportSuccess()
fun showDownloadSuccess()
fun showError()
fun cancelProgress()
}
interface ImportFlowOperations {
fun updateExtractionProgress(progress: Float)
fun markImportPipelineSuccess()
fun clearImportState()
fun getAndClearImportedFiles(): List<File>
}
interface ReindexingSteps {
suspend fun waitForIndexingCompletion(): Boolean
suspend fun extractArchives(onProgress: (Float) -> Unit)
suspend fun scanDirectory(path: String): OperationResult<Int>
suspend fun warmupIndexes()
suspend fun performAllCleanup()
}
class DefaultReindexingSteps(
private val localDictionaryRepository: LocalDictionaryRepository,
private val downloadDictionaryRepository: DictionaryRepository,
) : ReindexingSteps {
override suspend fun waitForIndexingCompletion(): Boolean {
if (!localDictionaryRepository.indexingProgress.first().isIndexing) return true
var waitCount = 0
while (localDictionaryRepository.indexingProgress.first().isIndexing && waitCount < MAX_POLL) {
delay(POLL_INTERVAL)
waitCount++
}
return waitCount < MAX_POLL
}
override suspend fun extractArchives(onProgress: (Float) -> Unit) {
downloadDictionaryRepository.extractArchives(onProgress = onProgress)
}
override suspend fun scanDirectory(path: String): OperationResult<Int> =
localDictionaryRepository.scanDirectory(path)
override suspend fun warmupIndexes() = localDictionaryRepository.warmupIndexes()
override suspend fun performAllCleanup() {
downloadDictionaryRepository.performAllCleanup()
}
private companion object {
const val MAX_POLL = 100
val POLL_INTERVAL = 200.milliseconds
}
}
@@ -13,7 +13,7 @@ object DateUtils {
return dateFormat.format(calendar.time)
}
fun extractDateFromFileName(fileName: String): String? {
val regex = Regex("""_(\d{6})\.(gz|dsl|idx|dsl\.dz|dsl\.idx|dsl\.gz)$""")
val regex = Regex("""_(\d{6})\.(gz|dsl|idx|idx\.fold|dsl\.dz|dsl\.idx|dsl\.idx\.fold|dsl\.gz)$""")
return regex.find(fileName)?.groupValues?.get(1)
}
fun isDateBefore(date1: String, date2: String): Boolean {
@@ -7,8 +7,9 @@ import androidx.core.app.*
import androidx.core.content.ContextCompat
import com.example.research.MainActivity
import com.example.research.R
import com.example.research.common.progress.NotificationPort
import com.example.research.feature.download.receiver.DownloadCancelReceiver
class NotificationHelper(private val context: Context) {
class NotificationHelper(private val context: Context) : NotificationPort {
companion object {
const val CHANNEL_ID = "download_progress_channel"
const val NOTIFICATION_ID = 1
@@ -64,6 +65,22 @@ class NotificationHelper(private val context: Context) {
android.Manifest.permission.POST_NOTIFICATIONS
) == android.content.pm.PackageManager.PERMISSION_GRANTED
}
override fun showProgress(percent: Int, titleRes: Int) {
showUnifiedProgressNotification(
title = context.getString(titleRes),
contentText = "$percent%",
progressPercent = percent,
)
}
override fun showDownloadSuccess() = showSuccessNotification()
override fun showImportSuccess() = showImportSuccessNotification()
override fun showError() = showErrorNotification()
override fun cancelProgressNotification() = cancelNotification()
fun showSuccessNotification() {
if (!canShowNotification()) return
lastNotificationKey = 0
@@ -93,6 +110,8 @@ class NotificationHelper(private val context: Context) {
notificationManager.cancel(NOTIFICATION_ID)
}
override fun hasActiveProgressNotification(): Boolean = lastNotificationKey != 0
fun showUnifiedProgressNotification(
title: String,
contentText: String,
@@ -13,7 +13,8 @@ data class IndexEntry(
val offset: ArticleOffset = ArticleOffset.ZERO,
val length: ArticleLength = ArticleLength.ZERO,
val dictionaryName: String = "",
val dictionaryPath: String = ""
val dictionaryPath: String = "",
val isAlias: Boolean = false
) : Comparable<IndexEntry> {
init {
require(word.isNotBlank()) { "Word cannot be blank" }
@@ -1,6 +1,8 @@
package com.example.research.core.domain.model
import kotlin.math.roundToInt
import androidx.compose.runtime.Immutable
/**
* Snapshot of an ongoing indexing operation.
*
@@ -11,6 +13,7 @@ import kotlin.math.roundToInt
* [perFileProgress] is kept for diagnostics; callers should prefer
* [progress] / [progressPercent].
*/
@Immutable
data class IndexingProgress(
val currentFile: String = "",
val currentIndex: Int = 0,
@@ -0,0 +1,27 @@
package com.example.research.core.domain.usecase
import com.example.research.core.domain.model.Dictionary
import com.example.research.core.domain.model.DictionarySource
import java.io.File
object DictionarySourceFileMatcher {
fun matches(source: DictionarySource, fileName: String): Boolean =
DictionarySource.matchesDictionaryFile(source.urlTemplate, fileName)
fun matches(source: DictionarySource, dictionary: Dictionary): Boolean =
matches(source, File(dictionary.path).name)
fun installedSources(
sources: List<DictionarySource>,
dictionaries: List<Dictionary>,
): List<DictionarySource> = sources.filter { source ->
dictionaries.any { dictionary -> matches(source, dictionary) }
}
fun installedSourcesForFileNames(
sources: List<DictionarySource>,
fileNames: Collection<String>,
): List<DictionarySource> = sources.filter { source ->
fileNames.any { fileName -> matches(source, fileName) }
}
}
@@ -43,12 +43,4 @@ class DictionarySourceValidator {
return ValidationResult.Valid
}
fun findMatchingSourceWithPrecomputedPrefixes(
dictionaryPrefix: String,
sourcesWithPrefixes: List<Pair<DictionarySource, String?>>
): DictionarySource? {
return sourcesWithPrefixes.find { (_, sourcePrefix) ->
sourcePrefix != null && dictionaryPrefix.equals(sourcePrefix, ignoreCase = true)
}?.first
}
}
@@ -30,17 +30,23 @@ class ManageDictionarySourcesUseCase(
preferencesManager.removeDictionarySource(sourceId)
}
suspend fun removeSourceForDictionary(dictionary: Dictionary) {
val dictionaryPrefix = dictionary.name
val sources = preferencesManager.dictionarySources.first()
val sourcesWithPrefixes = sources.map { source ->
source to DictionarySource.extractPrefix(source.urlTemplate)
}
suspend fun removeSourceForDictionary(
dictionary: Dictionary,
remainingDictionaryFileNames: Collection<String>?,
) {
if (remainingDictionaryFileNames == null) return
val matchingSource = validator.findMatchingSourceWithPrecomputedPrefixes(dictionaryPrefix, sourcesWithPrefixes)
if (matchingSource != null) {
preferencesManager.removeDictionarySource(matchingSource.id)
}
val sources = preferencesManager.dictionarySources.first()
val sourceIdsToRemove = sources
.filter { source ->
DictionarySourceFileMatcher.matches(source, dictionary) &&
remainingDictionaryFileNames.none { fileName ->
DictionarySourceFileMatcher.matches(source, fileName)
}
}
.map(DictionarySource::id)
preferencesManager.removeDictionarySources(sourceIdsToRemove)
}
sealed class AddSourceResult {
@@ -40,6 +40,10 @@ fun Throwable.isNetworkError(): Boolean {
if (isSslHandshakeError()) return true
return when (this) {
is java.net.UnknownHostException,
is java.net.ConnectException,
is java.net.NoRouteToHostException,
is java.net.BindException -> true
is java.io.IOException -> {
val message = message ?: ""
message.contains("connection", ignoreCase = true) ||
@@ -49,9 +53,14 @@ fun Throwable.isNetworkError(): Boolean {
message.contains("unreachable", ignoreCase = true) ||
message.contains("no route", ignoreCase = true) ||
message.contains("broken pipe", ignoreCase = true) ||
message.contains("resolve host", ignoreCase = true) ||
message.contains("unexpected end of stream", ignoreCase = true) ||
message.contains("socket closed", ignoreCase = true) ||
message.contains("ECONNRESET", ignoreCase = true) ||
message.contains("ECONNREFUSED", ignoreCase = true) ||
message.contains("ENETUNREACH", ignoreCase = true)
message.contains("ECONNABORTED", ignoreCase = true) ||
message.contains("ENETUNREACH", ignoreCase = true) ||
message.contains("EHOSTUNREACH", ignoreCase = true)
}
else -> false
}
@@ -9,6 +9,32 @@ fun String.sanitizeQuery(): String {
.take(MAX_QUERY_LENGTH)
}
fun String.foldLatinDiacritics(): String {
val decomposed = java.text.Normalizer.normalize(this, java.text.Normalizer.Form.NFD)
val folded = StringBuilder(decomposed.length)
var followsLatinBase = false
var index = 0
while (index < decomposed.length) {
val codePoint = decomposed.codePointAt(index)
val type = Character.getType(codePoint)
val isMark = type == Character.NON_SPACING_MARK.toInt() ||
type == Character.COMBINING_SPACING_MARK.toInt() ||
type == Character.ENCLOSING_MARK.toInt()
if (!(isMark && followsLatinBase)) {
folded.appendCodePoint(codePoint)
}
if (!isMark) {
followsLatinBase = Character.UnicodeScript.of(codePoint) == Character.UnicodeScript.LATIN
}
index += Character.charCount(codePoint)
}
return java.text.Normalizer.normalize(folded, java.text.Normalizer.Form.NFC)
}
fun String.sanitizeCacheKey(): String {
return this.replace(":", "_").replace("\n", "_").replace("\t", "_")
}
@@ -6,6 +6,7 @@ import androidx.datastore.preferences.core.Preferences
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.booleanPreferencesKey
import androidx.datastore.preferences.core.stringPreferencesKey
import androidx.datastore.preferences.core.stringSetPreferencesKey
import androidx.datastore.preferences.preferencesDataStore
import com.example.research.core.domain.model.DictionarySource
import kotlinx.coroutines.flow.Flow
@@ -23,6 +24,8 @@ class PreferencesManager(private val context: Context) {
val IS_THEME_EXPANDED = booleanPreferencesKey("is_theme_expanded")
val IS_LANGUAGE_EXPANDED = booleanPreferencesKey("is_language_expanded")
val IS_DICTIONARIES_EXPANDED = booleanPreferencesKey("is_dictionaries_expanded")
val DISABLED_DICTIONARY_PATHS = stringSetPreferencesKey("disabled_dictionary_paths")
val HAD_NO_DICTIONARIES = booleanPreferencesKey("had_no_dictionaries")
}
private val json = Json { ignoreUnknownKeys = true }
@@ -60,6 +63,44 @@ class PreferencesManager(private val context: Context) {
}
}
val disabledDictionaryPaths: Flow<Set<String>> = context.dataStore.data
.map { preferences ->
preferences[PreferencesKeys.DISABLED_DICTIONARY_PATHS] ?: emptySet()
}
val hadNoDictionaries: Flow<Boolean> = context.dataStore.data
.map { preferences ->
preferences[PreferencesKeys.HAD_NO_DICTIONARIES] ?: true
}
suspend fun setHadNoDictionaries(value: Boolean) {
context.dataStore.edit { preferences ->
preferences[PreferencesKeys.HAD_NO_DICTIONARIES] = value
}
}
suspend fun setDictionaryActive(path: String, isActive: Boolean) {
context.dataStore.edit { preferences ->
val disabledPaths = preferences[PreferencesKeys.DISABLED_DICTIONARY_PATHS]
.orEmpty()
.toMutableSet()
if (isActive) {
disabledPaths.remove(path)
} else {
disabledPaths.add(path)
}
if (disabledPaths.isEmpty()) {
preferences.remove(PreferencesKeys.DISABLED_DICTIONARY_PATHS)
} else {
preferences[PreferencesKeys.DISABLED_DICTIONARY_PATHS] = disabledPaths
}
}
}
suspend fun removeDictionaryActiveState(path: String) {
setDictionaryActive(path, true)
}
suspend fun saveThemeMode(theme: String) {
context.dataStore.edit { preferences ->
preferences[PreferencesKeys.THEME_MODE] = theme
@@ -15,85 +15,180 @@ import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.CancellationException
import java.io.File
/**
* Two-phase search paging.
*
* Page one (key = null) is the ranked head: the capped, relevance-ordered result the
* search screen has always shown, merged across active dictionaries. Every range scan
* that hit its cap hands back a [IndexSearcher.RangeCursor]; those become the tail key.
*
* Tail pages continue the unfinished range scans directly from the index files and
* merge them alphabetically (k-way, by index word), so scrolling eventually surfaces
* every prefix match in every active dictionary. Articles already shown -- by the head
* or by an earlier tail page, including diacritic alias duplicates -- are filtered
* through [emittedArticles], which grows only as far as the user actually scrolls.
*
* A new query builds a new PagingSource (see SearchViewModel's flatMapLatest), so an
* in-flight head or tail load is cancelled by Paging when the user keeps typing.
*/
class IndexEntryPagingSource(
private val indexSearcher: IndexSearcher,
private val dictionaries: List<Dictionary>,
private val query: String
) : PagingSource<Int, IndexEntry>() {
) : PagingSource<IndexEntryPagingSource.TailKey, IndexEntry>() {
private var searchResults: List<IndexEntry>? = null
data class StreamCursor(
val dictPosition: Int,
val cursor: IndexSearcher.RangeCursor
)
override val jumpingSupported: Boolean = true
data class TailKey(val streams: List<StreamCursor>)
override suspend fun load(params: LoadParams<Int>): LoadResult<Int, IndexEntry> {
val offset = params.key ?: 0
private data class EmittedKey(
val dictPosition: Int,
val word: String,
val offset: Long
)
private val emittedArticles = HashSet<EmittedKey>()
private val activeDictionaries = dictionaries.withIndex().filter { it.value.isActive }
override suspend fun load(params: LoadParams<TailKey>): LoadResult<TailKey, IndexEntry> {
return try {
val allResults = searchResults ?: ReSearchTrace.asyncSection(ReSearchTrace.SEARCH_PAGING) {
coroutineScope {
val rawResults = dictionaries
.filter { it.isActive }
.map { dict ->
async {
val indexFile = File(dict.indexPath)
if (!indexFile.exists()) return@async emptyList()
try {
val results = indexSearcher.search(
pathOrUri = indexFile.absolutePath,
query = query,
includeSubstringMatches = false
)
results.map {
it.withDictionary(
dict.metadata.name.ifBlank { dict.name },
dict.path
)
}
} catch (e: CancellationException) {
throw e
} catch (_: Exception) {
emptyList()
}
}
}
.awaitAll()
.flatten()
// Precompute sorting keys to optimize comparator performance.
// entry.word is already lowercase from indexing.
val normalizedQuery = query.sanitizeQuery().lowercase().trim()
rawResults.rankBySearchRelevance(
SearchRankingContext(normalizedQuery)
)
}
}.also { searchResults = it }
val totalSize = allResults.size
val startIndex = offset.coerceAtLeast(0).coerceAtMost(totalSize)
val endIndex = (startIndex + params.loadSize).coerceAtMost(totalSize)
val pageItems = allResults.subList(startIndex, endIndex)
val hasMore = endIndex < totalSize
val nextKey = if (hasMore) endIndex else null
val prevKey = if (offset == 0) null else (offset - params.loadSize).coerceAtLeast(0)
LoadResult.Page(
data = pageItems,
prevKey = prevKey,
nextKey = nextKey
)
val key = params.key
if (key == null) loadHead() else loadTail(key, params.loadSize)
} catch (e: CancellationException) {
throw e
} catch (e: Exception) {
LoadResult.Error(e)
}
}
override fun getRefreshKey(state: PagingState<Int, IndexEntry>): Int? {
return state.anchorPosition?.let { anchorPosition ->
state.closestPageToPosition(anchorPosition)?.prevKey?.plus(state.config.pageSize)
?: state.closestPageToPosition(anchorPosition)?.nextKey?.minus(state.config.pageSize)
private suspend fun loadHead(): LoadResult<TailKey, IndexEntry> =
ReSearchTrace.asyncSection(ReSearchTrace.SEARCH_PAGING) {
coroutineScope {
val perDictionary = activeDictionaries
.map { (position, dict) ->
async {
val indexFile = File(dict.indexPath)
if (!indexFile.exists()) return@async null
try {
val result = indexSearcher.searchWithCursors(
pathOrUri = indexFile.absolutePath,
query = query
)
Triple(position, dict, result)
} catch (e: CancellationException) {
throw e
} catch (_: Exception) {
null
}
}
}
.awaitAll()
.filterNotNull()
val rawResults = perDictionary.flatMap { (position, dict, result) ->
result.entries.map { entry ->
position to entry.withDictionary(
dict.metadata.name.ifBlank { dict.name },
dict.path
)
}
}
val normalizedQuery = query.sanitizeQuery().lowercase().trim()
val ranked = rawResults.map { it.second }
.rankBySearchRelevance(SearchRankingContext(normalizedQuery))
.distinctBy { Triple(it.dictionaryPath, it.word, it.offset.value) }
rawResults.forEach { (position, entry) -> emittedArticles.add(entry.emittedKey(position)) }
val streams = perDictionary.flatMap { (position, _, result) ->
result.cursors.map { StreamCursor(position, it) }
}
LoadResult.Page(
data = ranked,
prevKey = null,
nextKey = if (streams.isEmpty()) null else TailKey(streams)
)
}
}
private suspend fun loadTail(key: TailKey, loadSize: Int): LoadResult<TailKey, IndexEntry> {
val streams = key.streams.map { TailStream(it.dictPosition, it.cursor) }
val page = ArrayList<IndexEntry>(loadSize)
val pageKeys = HashSet<EmittedKey>()
val chunkSize = maxOf(loadSize, MIN_CHUNK)
while (page.size < loadSize) {
streams.forEach { it.ensureBuffered(chunkSize) }
val next = streams
.filter { it.hasBuffered }
.minByOrNull { it.peekWord }
?: break
val scanned = next.consume()
val dict = dictionaries[next.dictPosition]
val entry = scanned.entry.withDictionary(
dict.metadata.name.ifBlank { dict.name },
dict.path
)
val emittedKey = entry.emittedKey(next.dictPosition)
if (emittedKey !in emittedArticles && pageKeys.add(emittedKey)) {
page.add(entry)
}
}
emittedArticles.addAll(pageKeys)
val remaining = streams.mapNotNull { stream ->
stream.nextCursor?.let { StreamCursor(stream.dictPosition, it) }
}
return LoadResult.Page(
data = page,
prevKey = null,
nextKey = if (remaining.isEmpty()) null else TailKey(remaining)
)
}
private inner class TailStream(
val dictPosition: Int,
initialCursor: IndexSearcher.RangeCursor
) {
var nextCursor: IndexSearcher.RangeCursor? = initialCursor
private set
private var buffer: List<IndexSearcher.ScannedEntry> = emptyList()
private var position = 0
val hasBuffered: Boolean get() = position < buffer.size
val peekWord: String get() = buffer[position].entry.word
suspend fun ensureBuffered(chunkSize: Int) {
if (hasBuffered) return
val cursor = nextCursor ?: return
val indexPath = File(dictionaries[dictPosition].indexPath).absolutePath
buffer = indexSearcher.scanTailChunk(indexPath, cursor, chunkSize)
position = 0
if (buffer.isEmpty()) nextCursor = null
}
fun consume(): IndexSearcher.ScannedEntry {
val scanned = buffer[position]
position++
nextCursor = scanned.cursorAfter
return scanned
}
}
private fun IndexEntry.emittedKey(dictPosition: Int) =
EmittedKey(dictPosition, word, offset.value)
override fun getRefreshKey(state: PagingState<TailKey, IndexEntry>): TailKey? {
return null
}
}
private const val MIN_CHUNK = 48
@@ -58,17 +58,23 @@ object DslCharsetDetector {
private fun looksLikeUtf8(inputStream: java.io.InputStream): Boolean {
if (!inputStream.markSupported()) return false
inputStream.mark(8192)
return try {
try {
val sample = ByteArray(8192)
val read = inputStream.read(sample)
if (read <= 0) return false
val decoder = Charsets.UTF_8.newDecoder()
.onMalformedInput(java.nio.charset.CodingErrorAction.REPORT)
.onUnmappableCharacter(java.nio.charset.CodingErrorAction.REPORT)
decoder.decode(java.nio.ByteBuffer.wrap(sample, 0, read))
true
val byteBuffer = java.nio.ByteBuffer.wrap(sample, 0, read)
val charBuffer = java.nio.CharBuffer.allocate(8192)
while (true) {
val result = decoder.decode(byteBuffer, charBuffer, false)
if (result.isError) return false
if (result.isUnderflow) return true
charBuffer.clear()
}
} catch (_: Exception) {
false
return false
} finally {
inputStream.reset()
}
@@ -24,10 +24,11 @@ object DslHeadwordParser {
}
if (trimmed.indexOf('{') == -1 && trimmed.indexOf('[') == -1) {
val unescaped = unescape(trimmed)
return ParsedHeadword(
simplified = trimmed,
displayText = trimmed,
searchableText = trimmed.lowercase()
simplified = unescaped,
displayText = unescaped,
searchableText = unescaped.lowercase()
)
}
@@ -60,12 +61,12 @@ object DslHeadwordParser {
val searchableText = createSearchableText(trimmed)
return ParsedHeadword(
simplified = simplified,
displayText = displayText,
searchableText = searchableText
simplified = unescape(simplified),
displayText = unescape(displayText),
searchableText = unescape(searchableText)
)
}
private fun buildDisplayText(
reconstructedPrefix: String,
simplified: String,
@@ -96,7 +97,7 @@ object DslHeadwordParser {
.orEmpty()
}
}
private fun createSearchableText(originalHeadword: String): String {
return removeFormattingTags(
removeCurlyBraceMatches(originalHeadword, removeEmpty = true)
@@ -129,8 +130,17 @@ object DslHeadwordParser {
if (value.indexOf('{') == -1 && value.indexOf('}') == -1) return value
return buildString(value.length) {
value.forEach { char ->
if (char != '{' && char != '}') append(char)
var index = 0
while (index < value.length) {
val char = value[index]
if (char == '\\' && index + 1 < value.length) {
append(char)
append(value[index + 1])
index += 2
} else {
if (char != '{' && char != '}') append(char)
index++
}
}
}
}
@@ -138,6 +148,10 @@ object DslHeadwordParser {
private fun firstFormattingTagIndex(value: String): Int {
var index = 0
while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
continue
}
if (value[index] == '[' && formattingTagEnd(value, index) > index) {
return index
}
@@ -149,8 +163,12 @@ object DslHeadwordParser {
private fun firstCurlyContent(value: String): String? {
var index = 0
while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
continue
}
if (value[index] == '{') {
val end = value.indexOf('}', startIndex = index + 1)
val end = matchingCurlyEnd(value, index + 1)
if (end > index + 1) {
return value.substring(index + 1, end)
}
@@ -161,7 +179,7 @@ object DslHeadwordParser {
}
private fun substringBeforeFirstBracket(value: String): String {
val bracketIndex = value.indexOf('[')
val bracketIndex = indexOfUnescaped(value, '[')
return if (bracketIndex >= 0) value.substring(0, bracketIndex) else value
}
@@ -171,8 +189,12 @@ object DslHeadwordParser {
var index = 0
while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
continue
}
if (value[index] == '{') {
val end = value.indexOf('}', startIndex = index + 1)
val end = matchingCurlyEnd(value, index + 1)
if (end >= 0 && (removeEmpty || end > index + 1)) {
if (builder == null) {
builder = StringBuilder(value.length)
@@ -197,6 +219,10 @@ object DslHeadwordParser {
var index = 0
while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
continue
}
if (value[index] == '[') {
val tagEnd = formattingTagEnd(value, index)
if (tagEnd > index) {
@@ -217,6 +243,56 @@ object DslHeadwordParser {
}?.toString() ?: value
}
private fun matchingCurlyEnd(value: String, startIndex: Int): Int {
var index = startIndex
while (index < value.length) {
when {
value[index] == '\\' && index + 1 < value.length -> index += 2
value[index] == '{' -> return -1
value[index] == '}' -> return index
else -> index++
}
}
return -1
}
private fun indexOfUnescaped(
value: String,
target: Char,
startIndex: Int = 0,
): Int {
var index = startIndex.coerceAtLeast(0)
while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
index += 2
} else if (value[index] == target) {
return index
} else {
index++
}
}
return -1
}
private fun unescape(value: String): String {
val firstEscape = value.indexOf('\\')
if (firstEscape < 0) return value
return buildString(value.length) {
append(value, 0, firstEscape)
var index = firstEscape
while (index < value.length) {
if (value[index] == '\\' && index + 1 < value.length) {
append(value[index + 1])
index += 2
} else {
append(value[index])
index++
}
}
}
}
private fun formattingTagEnd(value: String, openBracketIndex: Int): Int {
val tokenStart = openBracketIndex + 1
if (tokenStart >= value.length) return -1
@@ -4,6 +4,7 @@ import android.util.Log
import com.example.research.core.domain.model.ArticleLength
import com.example.research.core.domain.model.ArticleOffset
import com.example.research.core.domain.model.IndexEntry
import com.example.research.core.util.foldLatinDiacritics
import com.example.research.core.util.removeAccentTagsForIndexing
import kotlinx.coroutines.Deferred
import kotlinx.coroutines.Dispatchers
@@ -23,7 +24,7 @@ class DslIndexer(context: Context? = null) {
companion object {
private const val TAG = "DslIndexer"
const val INDEX_VERSION = 4
const val INDEX_VERSION = 6
const val BUFFER_SIZE = 1048576
const val BATCH_BUFFER_SIZE = 524288
private const val INDEX_SORT_PARALLELISM = 1
@@ -40,6 +41,9 @@ class DslIndexer(context: Context? = null) {
private const val IN_MEMORY_SORT_MAX_ENTRIES = 140_000
private const val ESTIMATED_IN_MEMORY_ENTRY_BYTES = 768L
private const val IN_MEMORY_SORT_HEAP_RESERVE_BYTES = 96L * 1024L * 1024L
fun foldedIndexFile(indexFile: File): File =
File(indexFile.parent ?: ".", "${indexFile.name}.fold")
}
suspend fun createIndex(
@@ -56,21 +60,25 @@ class DslIndexer(context: Context? = null) {
java.io.BufferedInputStream(dslInputStream)
}
val tempFile = File(indexFile.parent, "${indexFile.name}.tmp")
val entryCount: Int
val counts: ParsedCounts
try {
try { foldedIndexFile(indexFile).delete() } catch (_: Throwable) { /* Legacy v5 leftover */ }
onOperationChange("Parsing file")
onFileProgress(0.0f)
entryCount = parseDslToTempFile(bufferedStream, tempFile,
counts = parseDslToTempFile(bufferedStream, tempFile,
onProgress, onFileProgress, onCharsetDetected)
onFileProgress(0.5f)
onOperationChange("Sorting index")
sortAndWriteIndexFile(tempFile, indexFile, entryCount, onOperationChange, onFileProgress)
sortAndWriteIndexFile(tempFile, indexFile, counts, onOperationChange, onFileProgress)
onFileProgress(1f)
} finally {
try { tempFile.delete() } catch (_: Throwable) { /* Ignore cleanup failure */ }
}
try { onProgress(entryCount) } catch (_: Throwable) { /* Ignore progress callback failure */ }
return entryCount
try { onProgress(counts.articleCount) } catch (_: Throwable) { /* Ignore progress callback failure */ }
return counts.articleCount
}
private data class ParsedCounts(val articleCount: Int, val totalEntries: Int)
// Blocking file I/O is safe across this file: every suspend fun here is reached only
// from createIndex(), which the sole caller (KotlinDictionaryEngine) runs inside
// withContext(Dispatchers.IO). The dispatcher is invisible across the suspend-call
@@ -83,7 +91,7 @@ class DslIndexer(context: Context? = null) {
onProgress: (Int) -> Unit,
onFileProgress: (Float) -> Unit = {},
onCharsetDetected: (DslCharsetDetector.DetectedCharset) -> Unit = {}
): Int {
): ParsedCounts {
val detectedCharset = DslCharsetDetector.detect(inputStream)
onCharsetDetected(detectedCharset)
@@ -98,6 +106,7 @@ class DslIndexer(context: Context? = null) {
)
var entryCount = 0
var aliasCount = 0
var lineCount = 0
var headwordLineCount = 0
var bodyLineCount = 0
@@ -127,9 +136,10 @@ class DslIndexer(context: Context? = null) {
fun normalizeSearchableWord(searchableText: String, fallback: String): String {
// Parser already lowercases and strips accent tags for non-empty results;
// only the raw-headword fallback still needs normalization.
return searchableText.ifEmpty {
val normalized = searchableText.ifEmpty {
fallback.removeAccentTagsForIndexing().lowercase()
}
return java.text.Normalizer.normalize(normalized, java.text.Normalizer.Form.NFC)
}
DataOutputStream(BufferedOutputStream(FileOutputStream(tempFile), BUFFER_SIZE)).use { output ->
@@ -144,12 +154,23 @@ class DslIndexer(context: Context? = null) {
return
}
val length = (groupEndOffset - groupStartOffset).coerceAtLeast(0L).toInt()
for (idx in groupSearchable.indices) {
writer.write(groupSearchable[idx])
writer.write(groupDisplay[idx])
fun writeTempEntry(word: String, display: String, isAlias: Boolean) {
writer.write(word)
writer.write(display)
output.writeLong(groupStartOffset)
output.writeInt(length)
output.writeByte(if (isAlias) ENTRY_FLAG_ALIAS else 0)
}
for (idx in groupSearchable.indices) {
val searchableWord = groupSearchable[idx]
val displayWord = groupDisplay[idx]
writeTempEntry(searchableWord, displayWord, isAlias = false)
entryCount++
val foldedWord = searchableWord.foldLatinDiacritics()
if (foldedWord != searchableWord) {
writeTempEntry(foldedWord, displayWord, isAlias = true)
aliasCount++
}
}
groupSearchable.clear()
groupDisplay.clear()
@@ -230,26 +251,28 @@ class DslIndexer(context: Context? = null) {
try { onProgress(entryCount) } catch (_: Throwable) { /* Ignore progress callback failure */ }
}
onFileProgress(0.5f)
return entryCount
return ParsedCounts(articleCount = entryCount, totalEntries = entryCount + aliasCount)
}
// Blocking file I/O safe: reached only from createIndex() (runs on Dispatchers.IO). See parseDslToTempFile().
@Suppress("BlockingMethodInNonBlockingContext")
private suspend fun sortAndWriteIndexFile(tempFile: File, indexFile: File, entryCount: Int, onOperationChange: (String) -> Unit = {}, onFileProgress: (Float) -> Unit = {}) {
if (entryCount == 0) {
private suspend fun sortAndWriteIndexFile(tempFile: File, indexFile: File, counts: ParsedCounts, onOperationChange: (String) -> Unit = {}, onFileProgress: (Float) -> Unit = {}) {
if (counts.totalEntries == 0) {
DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile))).use { output ->
output.writeInt(INDEX_VERSION)
output.writeInt(0)
output.writeInt(0)
output.writeInt(0)
}
return
}
if (shouldUseInMemorySort(entryCount)) {
inMemorySortAndWrite(tempFile, indexFile, entryCount)
if (shouldUseInMemorySort(counts.totalEntries)) {
inMemorySortAndWrite(tempFile, indexFile, counts)
onFileProgress(FINAL_INDEX_DONE_PROGRESS)
} else {
externalSortAndWrite(tempFile, indexFile, entryCount, onOperationChange, onFileProgress)
externalSortAndWrite(tempFile, indexFile, counts, onOperationChange, onFileProgress)
}
}
private fun shouldUseInMemorySort(entryCount: Int): Boolean {
if (entryCount <= IN_MEMORY_SORT_ALWAYS_ENTRIES) return true
if (entryCount > IN_MEMORY_SORT_MAX_ENTRIES) return false
@@ -264,30 +287,22 @@ class DslIndexer(context: Context? = null) {
}
// Blocking file I/O safe: reached only from createIndex() (runs on Dispatchers.IO). See parseDslToTempFile().
@Suppress("BlockingMethodInNonBlockingContext")
private suspend fun inMemorySortAndWrite(tempFile: File, indexFile: File, entryCount: Int) {
val entries = ArrayList<IndexEntry>(entryCount)
private suspend fun inMemorySortAndWrite(tempFile: File, indexFile: File, counts: ParsedCounts) {
val entries = ArrayList<IndexEntry>(counts.totalEntries)
java.io.DataInputStream(java.io.BufferedInputStream(java.io.FileInputStream(tempFile), BATCH_BUFFER_SIZE)).use { input ->
repeat(entryCount) {
val word = input.readUtf8String()
val originalWord = input.readUtf8String()
val offset = input.readLong()
val length = input.readInt()
entries.add(IndexEntry(
word = word,
originalWord = originalWord,
offset = ArticleOffset(offset),
length = ArticleLength(length)
))
repeat(counts.totalEntries) {
entries.add(input.readTempEntry())
if (entries.size % 5000 == 0) {
try { yield() } catch (_: Throwable) { /* Ignore yield failure */ }
}
}
}
entries.sort()
writeIndexFile(indexFile, entries)
writeIndexFile(indexFile, entries, counts.articleCount)
entries.clear()
}
private suspend fun externalSortAndWrite(tempFile: File, indexFile: File, entryCount: Int, onOperationChange: (String) -> Unit, onFileProgress: (Float) -> Unit) {
private suspend fun externalSortAndWrite(tempFile: File, indexFile: File, counts: ParsedCounts, onOperationChange: (String) -> Unit, onFileProgress: (Float) -> Unit) {
val entryCount = counts.totalEntries
val adaptiveConfig = resourceManager?.calculateAdaptiveConfig(estimatedEntryCount = entryCount.toLong())
?: ResourceManager.AdaptiveConfig(
batchSize = 100_000,
@@ -317,16 +332,7 @@ class DslIndexer(context: Context? = null) {
val batch = ArrayList<IndexEntry>(currentBatchSize)
val batchEnd = minOf(processed + currentBatchSize, entryCount)
while (processed < batchEnd) {
val word = input.readUtf8String()
val originalWord = input.readUtf8String()
val offset = input.readLong()
val length = input.readInt()
batch.add(IndexEntry(
word = word,
originalWord = originalWord,
offset = ArticleOffset(offset),
length = ArticleLength(length)
))
batch.add(input.readTempEntry())
processed++
}
@@ -422,7 +428,7 @@ class DslIndexer(context: Context? = null) {
onFileProgress(MERGE_START_PROGRESS)
onOperationChange("Merging ${sortedBatches.size} batches")
mergeSortedBatches(sortedBatches, indexFile, entryCount, onOperationChange, onFileProgress)
mergeSortedBatches(sortedBatches, indexFile, counts, onOperationChange, onFileProgress)
onFileProgress(FINAL_INDEX_DONE_PROGRESS)
} finally {
sortedBatches.forEach { try { it.delete() } catch (_: Throwable) { /* Ignore cleanup failure */ } }
@@ -443,6 +449,7 @@ class DslIndexer(context: Context? = null) {
writer.write(entry.originalWord)
output.writeLong(entry.offset.value)
output.writeInt(entry.length.value)
output.writeByte(if (entry.isAlias) ENTRY_FLAG_ALIAS else 0)
}
}
}
@@ -453,15 +460,16 @@ class DslIndexer(context: Context? = null) {
private suspend fun mergeSortedBatches(
batchFiles: List<File>,
indexFile: File,
totalEntries: Int,
counts: ParsedCounts,
onOperationChange: (String) -> Unit = {},
onFileProgress: (Float) -> Unit,
) {
if (batchFiles.isEmpty()) return
val totalEntries = counts.totalEntries
val sparsePoints = mutableListOf<SparsePoint>()
val prefixCounts = HashMap<String, Int>(minOf(totalEntries / 10, 100_000))
val headerSize = 4 + 4 + 4
val headerSize = 4 + 4 + 4 + 4
val dataScratchFile = File(indexFile.parent, "${indexFile.name}.data")
val pq = PriorityQueue<BatchReader>()
@@ -508,7 +516,8 @@ class DslIndexer(context: Context? = null) {
val origSize = writer.write(entry.originalWord)
dataOutput.writeLong(entry.offset.value)
dataOutput.writeInt(entry.length.value)
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4
dataOutput.writeByte(if (entry.isAlias) ENTRY_FLAG_ALIAS else 0)
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 + 1
entryIndex++
@@ -542,6 +551,7 @@ class DslIndexer(context: Context? = null) {
DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile), BUFFER_SIZE)).use { output ->
output.writeInt(INDEX_VERSION)
output.writeInt(totalEntries)
output.writeInt(counts.articleCount)
output.writeInt(sparsePoints.size)
sparsePoints.forEachIndexed { idx, point ->
@@ -632,18 +642,14 @@ class DslIndexer(context: Context? = null) {
}
private fun readNextEntry(input: java.io.DataInputStream): IndexEntry? {
return try {
val word = input.readUtf8String()
val originalWord = input.readUtf8String()
val offset = ArticleOffset(input.readLong())
val length = ArticleLength(input.readInt())
IndexEntry(word, originalWord, offset, length)
input.readTempEntry()
} catch (_: java.io.EOFException) {
null
}
}
private fun writeIndexFile(indexFile: File, entries: List<IndexEntry>) {
private fun writeIndexFile(indexFile: File, entries: List<IndexEntry>, articleCount: Int) {
val sparseData = buildAdaptiveSparseIndex(entries)
val headerSize = 4 + 4 + 4
val headerSize = 4 + 4 + 4 + 4
// Cache sparse table bytes to avoid redundant conversions
val sparseTableBytes = sparseData.map { CachedBytes.from(it.word) }
@@ -657,13 +663,14 @@ class DslIndexer(context: Context? = null) {
// Calculate sizes without allocating byte arrays
val wordSize = utf8ByteSize(entry.word)
val origSize = utf8ByteSize(entry.originalWord)
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 + 1
}
DataOutputStream(BufferedOutputStream(FileOutputStream(indexFile))).use { output ->
val writer = Utf8Writer(output)
output.writeInt(INDEX_VERSION)
output.writeInt(entries.size)
output.writeInt(articleCount)
output.writeInt(sparseData.size)
sparseData.forEach { point ->
@@ -677,6 +684,7 @@ class DslIndexer(context: Context? = null) {
writer.write(entry.originalWord)
output.writeLong(entry.offset.value)
output.writeInt(entry.length.value)
output.writeByte(if (entry.isAlias) ENTRY_FLAG_ALIAS else 0)
}
}
}
@@ -707,7 +715,7 @@ class DslIndexer(context: Context? = null) {
// Calculate sizes without allocating byte arrays
val wordSize = utf8ByteSize(entry.word)
val origSize = utf8ByteSize(entry.originalWord)
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4
currentByteOffset += 2 + wordSize + 2 + origSize + 8 + 4 + 1
}
return sparsePoints
@@ -824,6 +832,24 @@ private fun java.io.DataInputStream.readUtf8String(): String {
readFully(bytes)
return String(bytes, Charsets.UTF_8)
}
internal const val ENTRY_FLAG_ALIAS = 1
private fun java.io.DataInputStream.readTempEntry(): IndexEntry {
val word = readUtf8String()
val originalWord = readUtf8String()
val offset = readLong()
val length = readInt()
val flags = readUnsignedByte()
return IndexEntry(
word = word,
originalWord = originalWord,
offset = ArticleOffset(offset),
length = ArticleLength(length),
isAlias = flags and ENTRY_FLAG_ALIAS != 0
)
}
private enum class LineKind {
HEADWORD,
BODY,
@@ -5,6 +5,7 @@ import com.example.research.R
import com.example.research.core.domain.model.ArticleLength
import com.example.research.core.domain.model.ArticleOffset
import com.example.research.core.domain.model.IndexEntry
import com.example.research.core.util.foldLatinDiacritics
import com.example.research.core.util.DictionaryError
import com.example.research.core.util.sanitizeCacheKey
import com.example.research.core.util.sanitizeQuery
@@ -23,7 +24,8 @@ import java.io.RandomAccessFile
class IndexSearcher(private val context: Context) {
private val metadataCache = MetadataLruCache(maxBytes = 64L * 1024L * 1024L)
private val resultsCache = ResultsLruCache(maxEntries = 20)
private val resultsCache = ResultsLruCache<List<IndexEntry>>(maxEntries = 20)
private val cursorResultsCache = ResultsLruCache<CursorSearchResult>(maxEntries = 20)
/**
* Pre-allocated reusable buffer for readEntry() to reduce allocations.
@@ -43,13 +45,26 @@ class IndexSearcher(private val context: Context) {
val dataStartOffset: Long
)
data class IndexComparisonSummary(
val expectedCount: Int,
val actualCount: Int,
val comparedCount: Int
data class RangeCursor(
val rangeQuery: String,
val absoluteIndex: Int,
val fileOffset: Long
)
class IndexComparisonException(message: String) : IllegalStateException(message)
internal class RangeScanResult(
val entries: List<IndexEntry>,
val nextCursor: RangeCursor?
)
class CursorSearchResult(
val entries: List<IndexEntry>,
val cursors: List<RangeCursor>
)
class ScannedEntry(
val entry: IndexEntry,
val cursorAfter: RangeCursor?
)
suspend fun findFirstEntry(pathOrUri: String, query: String): IndexEntry? = withContext(Dispatchers.Default) {
val normalizedQuery = query.sanitizeQuery().lowercase().trim()
@@ -66,7 +81,7 @@ class IndexSearcher(private val context: Context) {
var iterationCount = 0
while (currentIdx < count) {
if (iterationCount % 1000 == 0) yield()
val nextEntry = readEntry(source)
val nextEntry = readEntry(source, metadata)
val nextWordLower = nextEntry.word.lowercase()
if (!nextWordLower.startsWith(normalizedQuery)) break
if (nextWordLower == normalizedQuery) return@withContext nextEntry.copy(word = nextWordLower)
@@ -93,6 +108,7 @@ class IndexSearcher(private val context: Context) {
return@withContext emptyList()
}
val normalizedQuery = query.sanitizeQuery().lowercase().trim()
val foldedQuery = normalizedQuery.foldLatinDiacritics()
val cacheKey = "$pathOrUri:${normalizedQuery.sanitizeCacheKey()}"
resultsCache.get(cacheKey)?.let {
return@withContext it
@@ -102,33 +118,34 @@ class IndexSearcher(private val context: Context) {
val count = metadata.totalCount
if (count == 0) return@withContext emptyList()
val results = openSource(pathOrUri).use { source ->
val prefixMatches = mutableListOf<IndexEntry>()
val substringMatches = mutableListOf<IndexEntry>()
val firstMatch = findFirstMatch(source, metadata, normalizedQuery)
if (firstMatch != null) {
prefixMatches.addAll(readMatchingEntries(source, count, firstMatch, normalizedQuery))
val prefixMatches = readPrefixMatches(source, metadata, normalizedQuery).entries.toMutableList()
if (foldedQuery != normalizedQuery) {
prefixMatches.addAll(readPrefixMatches(source, metadata, foldedQuery).entries)
}
val substringMatches = mutableListOf<IndexEntry>()
val seenArticles = prefixMatches.mapTo(HashSet()) { it.articleKey() }
if (includeSubstringMatches &&
count <= SUBSTRING_SCAN_MAX_ENTRIES &&
prefixMatches.size < 100 &&
seenArticles.size < MAX_SUBSTRING_RESULTS &&
normalizedQuery.length >= 2
) {
val prefixMatchWords = prefixMatches.map { it.word }.toSet()
source.seek(metadata.dataStartOffset)
for (i in 0 until count) {
if (i % 1000 == 0) yield()
if (substringMatches.size >= 100) break
val entry = readEntry(source)
if (substringMatches.size >= MAX_SUBSTRING_RESULTS) break
val entry = readEntry(source, metadata)
val entryWordLower = entry.word.lowercase()
if (entryWordLower.contains(normalizedQuery) && entryWordLower !in prefixMatchWords) {
if (entryWordLower.contains(normalizedQuery) &&
seenArticles.add(entry.articleKey())
) {
substringMatches.add(entry.copy(word = entryWordLower))
}
}
}
val combinedResults = prefixMatches + substringMatches
combinedResults.rankBySearchRelevance(
SearchRankingContext(normalizedQuery)
)
(prefixMatches + substringMatches)
.rankBySearchRelevance(SearchRankingContext(normalizedQuery, foldedQuery))
.distinctBy { it.articleKey() }
}
resultsCache.put(cacheKey, results)
results
@@ -148,99 +165,83 @@ class IndexSearcher(private val context: Context) {
metadataCache.getOrPut(pathOrUri) { loadMetadata(pathOrUri) }
}
suspend fun compareEntryGroups(
expectedIndexPath: String,
actualIndexPath: String,
onGroup: suspend (word: String, expected: List<IndexEntry>, actual: List<IndexEntry>) -> Unit
): IndexComparisonSummary = withContext(Dispatchers.IO) {
val expectedMetadata = loadMetadata(expectedIndexPath)
val actualMetadata = loadMetadata(actualIndexPath)
if (expectedMetadata.totalCount != actualMetadata.totalCount) {
throw IndexComparisonException(
"Index entry count differs: expected=${expectedMetadata.totalCount}, " +
"actual=${actualMetadata.totalCount}"
)
suspend fun searchWithCursors(
pathOrUri: String,
query: String
): CursorSearchResult = withContext(Dispatchers.Default) {
if (query.isBlank()) {
return@withContext CursorSearchResult(emptyList(), emptyList())
}
openSource(expectedIndexPath).use { expectedSource ->
openSource(actualIndexPath).use { actualSource ->
expectedSource.seek(expectedMetadata.dataStartOffset)
actualSource.seek(actualMetadata.dataStartOffset)
var expectedNext = readEntryOrNull(expectedSource, expectedMetadata.totalCount > 0)
var actualNext = readEntryOrNull(actualSource, actualMetadata.totalCount > 0)
var comparedCount = 0
while (expectedNext != null && actualNext != null) {
if (expectedNext.word != actualNext.word) {
throw IndexComparisonException(
"Index words differ at entry $comparedCount: " +
"expected=${expectedNext.word}, actual=${actualNext.word}"
)
}
val word = expectedNext.word
val expectedGroup = mutableListOf<IndexEntry>()
val actualGroup = mutableListOf<IndexEntry>()
while (true) {
val current = expectedNext ?: break
if (current.word != word) break
expectedGroup.add(current)
comparedCount++
expectedNext = readEntryOrNull(
expectedSource,
comparedCount < expectedMetadata.totalCount
)
}
var actualReadCount = comparedCount - expectedGroup.size
while (true) {
val current = actualNext ?: break
if (current.word != word) break
actualGroup.add(current)
actualReadCount++
actualNext = readEntryOrNull(
actualSource,
actualReadCount < actualMetadata.totalCount
)
}
if (expectedGroup.size != actualGroup.size) {
throw IndexComparisonException(
"Headword multiplicity differs for '$word': " +
"expected=${expectedGroup.size}, actual=${actualGroup.size}"
)
}
onGroup(word, expectedGroup, actualGroup)
if (comparedCount % 10_000 == 0) yield()
}
if (expectedNext != null || actualNext != null) {
throw IndexComparisonException(
"One index ended before the other at entry $comparedCount"
)
}
IndexComparisonSummary(
expectedCount = expectedMetadata.totalCount,
actualCount = actualMetadata.totalCount,
comparedCount = comparedCount
)
val normalizedQuery = query.sanitizeQuery().lowercase().trim()
val foldedQuery = normalizedQuery.foldLatinDiacritics()
val cacheKey = "$pathOrUri:${normalizedQuery.sanitizeCacheKey()}"
cursorResultsCache.get(cacheKey)?.let {
return@withContext it
}
try {
val metadata = metadataCache.getOrPut(pathOrUri) { loadMetadata(pathOrUri) }
if (metadata.totalCount == 0) {
return@withContext CursorSearchResult(emptyList(), emptyList())
}
val result = openSource(pathOrUri).use { source ->
val scans = buildList {
add(readPrefixMatches(source, metadata, normalizedQuery))
if (foldedQuery != normalizedQuery) {
add(readPrefixMatches(source, metadata, foldedQuery))
}
}
val entries = scans.flatMap { it.entries }
.rankBySearchRelevance(SearchRankingContext(normalizedQuery, foldedQuery))
.distinctBy { it.articleKey() }
CursorSearchResult(entries, scans.mapNotNull { it.nextCursor })
}
cursorResultsCache.put(cacheKey, result)
result
} catch (e: CancellationException) {
throw e
} catch (e: Exception) {
Log.e(TAG, "searchWithCursors() - error: ${e.message}", e)
if (e is DictionaryError) throw e
throw DictionaryError.SearchError.QueryFailed(context.getString(R.string.error_search_failed, query.take(20), pathOrUri.take(20)), e)
}
}
private fun readEntryOrNull(source: RandomAccessSource, shouldRead: Boolean): IndexEntry? =
if (shouldRead) readEntry(source) else null
suspend fun scanTailChunk(
pathOrUri: String,
cursor: RangeCursor,
maxEntries: Int
): List<ScannedEntry> = withContext(Dispatchers.Default) {
val metadata = metadataCache.getOrPut(pathOrUri) { loadMetadata(pathOrUri) }
if (cursor.absoluteIndex >= metadata.totalCount) return@withContext emptyList()
openSource(pathOrUri).use { source ->
source.seek(cursor.fileOffset)
val out = ArrayList<ScannedEntry>(maxEntries)
var currentIdx = cursor.absoluteIndex
var iterationCount = 0
while (currentIdx < metadata.totalCount && out.size < maxEntries) {
if (iterationCount % 1000 == 0) yield()
val entry = readEntry(source, metadata)
val entryWordLower = entry.word.lowercase()
if (!entryWordLower.startsWith(cursor.rangeQuery)) break
currentIdx++
iterationCount++
val cursorAfter = if (currentIdx < metadata.totalCount) {
RangeCursor(cursor.rangeQuery, currentIdx, source.position())
} else {
null
}
out.add(ScannedEntry(entry.copy(word = entryWordLower), cursorAfter))
}
out
}
}
fun trimMemory(level: Int) {
metadataCache.trim(level)
resultsCache.trim(level)
cursorResultsCache.trim(level)
}
suspend fun isMetadataLoaded(pathOrUri: String): Boolean {
return metadataCache.contains(pathOrUri)
}
@@ -252,7 +253,13 @@ class IndexSearcher(private val context: Context) {
val count = input.readInt()
return@withContext if (version >= 2) {
loadMetadataV2(input, version, count)
val headerBase = if (version >= 6) {
input.readInt()
12L
} else {
8L
}
loadMetadataV2(input, version, count, headerBase)
} else {
loadMetadataV1(input, version, count)
}
@@ -272,13 +279,13 @@ class IndexSearcher(private val context: Context) {
// runs inside withContext(Dispatchers.IO). The dispatcher is not visible across the
// suspend-call boundary, so the inspection is suppressed deliberately.
@Suppress("BlockingMethodInNonBlockingContext")
private suspend fun loadMetadataV2(input: DataInputStream, version: Int, count: Int): IndexMetadata {
private suspend fun loadMetadataV2(input: DataInputStream, version: Int, count: Int, headerBase: Long): IndexMetadata {
val sparseCount = input.readInt()
val sparseIndices = IntArray(sparseCount)
val sparseOffsets = LongArray(sparseCount)
val sparseWords = Array(sparseCount) { "" }
var headerSize = 4 + 4 + 4L
var headerSize = headerBase + 4L
for (i in 0 until sparseCount) {
if (i % 1000 == 0) yield()
@@ -337,6 +344,17 @@ class IndexSearcher(private val context: Context) {
dataStartOffset = 8L
)
}
private suspend fun readPrefixMatches(
source: RandomAccessSource,
metadata: IndexMetadata,
query: String
): RangeScanResult {
val firstMatch = findFirstMatch(source, metadata, query)
?: return RangeScanResult(emptyList(), null)
return readMatchingEntries(source, metadata, firstMatch, query)
}
private suspend fun findFirstMatch(source: RandomAccessSource, metadata: IndexMetadata, query: String): Pair<Int, IndexEntry>? {
val sparseIndices = metadata.sparseIndices
val sparseOffsets = metadata.sparseOffsets
@@ -359,7 +377,7 @@ class IndexSearcher(private val context: Context) {
while (currentAbsoluteIdx < metadata.totalCount) {
if (iterationCount % 1000 == 0) yield()
val entry = readEntry(source)
val entry = readEntry(source, metadata)
val entryWordLower = entry.word.lowercase()
if (entryWordLower >= query) {
return if (entryWordLower.startsWith(query)) Pair(currentAbsoluteIdx, entry.copy(word = entryWordLower)) else null
@@ -394,53 +412,64 @@ class IndexSearcher(private val context: Context) {
}
private suspend fun readMatchingEntries(
source: RandomAccessSource,
totalCount: Int,
metadata: IndexMetadata,
firstMatch: Pair<Int, IndexEntry>,
query: String
): List<IndexEntry> {
): RangeScanResult {
val results = mutableListOf<IndexEntry>()
results.add(firstMatch.second)
val uniqueArticles = hashSetOf(firstMatch.second.articleKey())
var currentIdx = firstMatch.first + 1
var iterationCount = 0
while (currentIdx < totalCount && results.size < 100) {
while (currentIdx < metadata.totalCount) {
if (uniqueArticles.size >= MAX_PREFIX_RESULTS) {
return RangeScanResult(results, RangeCursor(query, currentIdx, source.position()))
}
if (iterationCount % 1000 == 0) yield()
val entry = readEntry(source)
val entry = readEntry(source, metadata)
val entryWordLower = entry.word.lowercase()
if (!entryWordLower.startsWith(query)) break
uniqueArticles.add(entry.articleKey())
results.add(entry.copy(word = entryWordLower))
currentIdx++
iterationCount++
}
return results
return RangeScanResult(results, null)
}
private fun readEntry(source: RandomAccessSource): IndexEntry {
val wordLen = source.readUnsignedShort()
val word = if (wordLen <= 4096) {
private fun readPooledString(source: RandomAccessSource, length: Int): String {
return if (length <= 4096) {
val buffer = entryBuffer.get() ?: throw IllegalStateException("Entry buffer pool exhausted")
source.readFully(buffer, 0, wordLen)
String(buffer, 0, wordLen, Charsets.UTF_8)
source.readFully(buffer, 0, length)
String(buffer, 0, length, Charsets.UTF_8)
} else {
val bytes = ByteArray(wordLen)
source.readFully(bytes)
String(bytes, Charsets.UTF_8)
}
val origLen = source.readUnsignedShort()
val originalWord = if (origLen <= 4096) {
val buffer = entryBuffer.get() ?: throw IllegalStateException("Entry buffer pool exhausted")
source.readFully(buffer, 0, origLen)
String(buffer, 0, origLen, Charsets.UTF_8)
} else {
val bytes = ByteArray(origLen)
val bytes = ByteArray(length)
source.readFully(bytes)
String(bytes, Charsets.UTF_8)
}
}
private fun readEntry(source: RandomAccessSource, metadata: IndexMetadata): IndexEntry {
val word = readPooledString(source, source.readUnsignedShort())
val originalWord = readPooledString(source, source.readUnsignedShort())
val offset = ArticleOffset(source.readLong())
val length = ArticleLength(source.readInt())
return IndexEntry(word, originalWord, offset, length)
val isAlias = if (metadata.version >= 6) {
source.readUnsignedByte() and ENTRY_FLAG_ALIAS != 0
} else {
false
}
return IndexEntry(word, originalWord, offset, length, isAlias = isAlias)
}
private data class ArticleKey(
val originalWord: String,
val offset: Long,
val length: Int
)
private fun IndexEntry.articleKey() = ArticleKey(originalWord, offset.value, length.value)
private fun openSource(pathOrUri: String): RandomAccessSource {
val file = File(pathOrUri)
if (!file.exists()) throw DictionaryError.FileSystemError.NotFound(pathOrUri)
@@ -455,20 +484,22 @@ class IndexSearcher(private val context: Context) {
private const val TAG = "IndexSearcher"
private const val SUBSTRING_SCAN_MAX_ENTRIES = 200_000
private const val SPARSE_INTERVAL_V1 = 128
private const val MAX_PREFIX_RESULTS = 200
private const val MAX_SUBSTRING_RESULTS = 100
}
}
private class ResultsLruCache(private val maxEntries: Int) {
private class ResultsLruCache<V : Any>(private val maxEntries: Int) {
private val mutex = Mutex()
private val map = object : LinkedHashMap<String, List<IndexEntry>>(maxEntries, 0.75f, true) {
override fun removeEldestEntry(eldest: MutableMap.MutableEntry<String, List<IndexEntry>>): Boolean {
private val map = object : LinkedHashMap<String, V>(maxEntries, 0.75f, true) {
override fun removeEldestEntry(eldest: MutableMap.MutableEntry<String, V>): Boolean {
return size > maxEntries
}
}
suspend fun get(key: String): List<IndexEntry>? = mutex.withLock {
suspend fun get(key: String): V? = mutex.withLock {
map[key]
}
suspend fun put(key: String, value: List<IndexEntry>) = mutex.withLock {
suspend fun put(key: String, value: V) = mutex.withLock {
map[key] = value
}
fun trim(level: Int) {
@@ -548,6 +579,8 @@ private class MetadataLruCache(private val maxBytes: Long) {
}
interface RandomAccessSource : Closeable {
fun seek(pos: Long)
fun position(): Long
fun readUnsignedByte(): Int
fun readUnsignedShort(): Int
fun readInt(): Int
fun readLong(): Long
@@ -561,6 +594,10 @@ class FileSource(file: File) : RandomAccessSource {
raf.seek(pos)
}
override fun position(): Long = raf.filePointer
override fun readUnsignedByte(): Int = raf.readUnsignedByte()
override fun readUnsignedShort(): Int = raf.readUnsignedShort()
override fun readInt(): Int = raf.readInt()
@@ -1,15 +1,17 @@
package com.example.research.data.parser
import com.example.research.core.domain.model.IndexEntry
import com.example.research.core.util.foldLatinDiacritics
internal data class SearchRankingContext(
val normalizedQuery: String,
val foldedQuery: String = normalizedQuery.foldLatinDiacritics(),
)
private data class SortKey(
val charCount: Int,
val matchTier: Int,
val exactMatch: Int,
val startsWithMatch: Int,
val charCount: Int,
val word: String,
val entry: IndexEntry,
)
@@ -26,13 +28,18 @@ internal fun List<IndexEntry>.rankBySearchRelevance(
.ifEmpty { entry.word.trim() }
SortKey(
charCount = displayKey.codePointCount(0, displayKey.length),
matchTier = when {
!entry.isAlias && word.startsWith(ranking.normalizedQuery) -> 0
entry.isAlias && word.startsWith(ranking.foldedQuery) -> 1
!entry.isAlias && word.foldLatinDiacritics().startsWith(ranking.foldedQuery) -> 1
else -> 2
},
exactMatch = if (word == ranking.normalizedQuery) 0 else 1,
startsWithMatch = if (word.startsWith(ranking.normalizedQuery)) 0 else 1,
charCount = displayKey.codePointCount(0, displayKey.length),
word = word,
entry = entry,
)
}
.sortedWith(compareBy({ it.charCount }, { it.exactMatch }, { it.startsWithMatch }, { it.word }))
.sortedWith(compareBy({ it.matchTier }, { it.exactMatch }, { it.charCount }, { it.word }))
.map { it.entry }
}
@@ -14,6 +14,7 @@ import com.example.research.core.util.extractDictionaryPrefix
import com.example.research.core.util.sanitizeQuery
import com.example.research.core.util.toDictionaryError
import com.example.research.data.dictzip.DictZipRandomAccessFile
import com.example.research.data.local.preferences.PreferencesManager
import com.example.research.data.parser.DslCharsetDetector
import com.example.research.data.parser.DslHeaderParser
import com.example.research.data.parser.DslIndexer
@@ -26,8 +27,9 @@ import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.ensureActive
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.flow.update
import kotlinx.coroutines.launch
import kotlinx.coroutines.supervisorScope
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.Semaphore
@@ -85,28 +87,23 @@ private class ProgressTracker(files: List<LocalDictionaryRepository.DiscoveredFi
class LocalDictionaryRepository(
private val context: Context,
private val engine: KotlinDictionaryEngine,
private val engine: KotlinDictionaryEngine = KotlinDictionaryEngine(context),
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO,
private val defaultDispatcher: CoroutineDispatcher = Dispatchers.Default,
private val preferencesManager: PreferencesManager = PreferencesManager(context),
) : CoroutineScope {
constructor(context: Context) : this(
context,
KotlinDictionaryEngine(context)
)
override val coroutineContext = Job() + ioDispatcher
private val mutableDictionaries = MutableStateFlow<List<Dictionary>>(emptyList())
val dictionaries: StateFlow<List<Dictionary>>
get() = mutableDictionaries
val dictionaries: StateFlow<List<Dictionary>> = mutableDictionaries.asStateFlow()
private val mutableIndexingProgress = MutableStateFlow(IndexingProgress())
val indexingProgress: StateFlow<IndexingProgress>
get() = mutableIndexingProgress
val indexingProgress: StateFlow<IndexingProgress> = mutableIndexingProgress.asStateFlow()
private val indexingMutex = Mutex()
private val indexingSemaphore = Semaphore(1)
private val warmupMutex = Mutex()
private val dictionaryStateMutex = Mutex()
@Volatile
private var currentIndexingJob: Job? = null
@@ -124,6 +121,24 @@ class LocalDictionaryRepository(
fun isIndexingInProgress(): Boolean = currentIndexingJob?.isActive == true
suspend fun listDictionaryPayloadFileNames(path: String): List<String>? =
withContext(ioDispatcher) {
try {
val directory = File(path)
when {
!directory.exists() -> {
if (directory.parentFile?.isDirectory == true) emptyList() else null
}
!directory.isDirectory -> null
else -> directory.listFiles { file -> isDictionaryPayloadFile(file) }
?.map(File::getName)
}
} catch (e: SecurityException) {
Log.w(TAG, "Failed to list dictionary files: ${e.message}")
null
}
}
suspend fun scanDirectory(pathOrUri: String): OperationResult<Int> = withContext(ioDispatcher) {
if (isIndexingInProgress()) {
return@withContext OperationResult.Error(
@@ -140,7 +155,7 @@ class LocalDictionaryRepository(
val filesToScan = scanLocalDirectory(pathOrUri)
if (filesToScan.isEmpty()) {
mutableDictionaries.value = emptyList()
replaceDictionaries(emptyList())
mutableIndexingProgress.value = IndexingProgress(isIndexing = false)
return@withContext OperationResult.Success(0)
}
@@ -159,12 +174,12 @@ class LocalDictionaryRepository(
}
if (filesToIndex.isEmpty()) {
mutableDictionaries.value = existingDictionaries.sortedBy { it.name }
replaceDictionaries(existingDictionaries)
engine.getIndexSearcher().trimMemory(80)
return@withContext OperationResult.Success(existingDictionaries.size)
}
mutableDictionaries.value = existingDictionaries.sortedBy { it.name }
replaceDictionaries(existingDictionaries)
val totalArticlesIndexed = AtomicInteger(existingDictionaries.sumOf { it.articleCount })
@@ -233,12 +248,9 @@ class LocalDictionaryRepository(
if (res is OperationResult.Success) {
progressTracker?.updateFileProgress(file.name, 1.0f)
totalArticlesIndexed.addAndGet(res.data.articleCount)
mutableDictionaries.update { current ->
(current + res.data)
.distinctBy { it.path }
.sortedBy { it.name }
}
res.data
val dictionary = res.data
addDictionary(dictionary)
dictionary
} else null
}
}
@@ -266,17 +278,37 @@ class LocalDictionaryRepository(
}
}
private suspend fun replaceDictionaries(dictionaries: List<Dictionary>) {
dictionaryStateMutex.withLock { applyDictionaries(dictionaries) }
}
private suspend fun addDictionary(dictionary: Dictionary) {
dictionaryStateMutex.withLock {
applyDictionaries((mutableDictionaries.value + dictionary).distinctBy { it.path })
}
}
private suspend fun applyDictionaries(dictionaries: List<Dictionary>) {
val disabledPaths = preferencesManager.disabledDictionaryPaths.first()
mutableDictionaries.value = dictionaries
.map { it.withActiveStatus(it.path !in disabledPaths) }
.sortedBy { it.name }
}
private suspend fun tryLoadExistingIndex(file: DiscoveredFile): Dictionary? {
val dictFile = File(file.localPath)
val indexFile = File(file.indexPath)
if (!indexFile.exists() || indexFile.length() == 0L) return null
if (!indexFile.exists() || indexFile.length() == 0L) {
deleteIndexFiles(indexFile)
return null
}
if (!dictFile.exists()) return null
val header = readIndexHeader(indexFile)
if (header == null || header.version != DslIndexer.INDEX_VERSION) {
try { indexFile.delete() } catch (_: Exception) {}
deleteIndexFiles(indexFile)
return null
}
@@ -326,23 +358,36 @@ class LocalDictionaryRepository(
)
OperationResult.Success(dict)
} else {
indexFile.delete()
deleteIndexFiles(indexFile)
OperationResult.Error(result.errorMsg, null)
}
} catch(e: Exception) {
indexFile.delete()
deleteIndexFiles(indexFile)
if (e is CancellationException) throw e
OperationResult.Error(e.message ?: context.getString(R.string.error_indexing), e)
}
}
private fun deleteIndexFiles(indexFile: File, errors: MutableList<String>? = null) {
deleteFileTracked(indexFile, "index file", errors)
deleteFileTracked(DslIndexer.foldedIndexFile(indexFile), "legacy folded index file", errors)
}
private fun deleteFileTracked(file: File, label: String, errors: MutableList<String>?) {
try {
if (file.exists() && !file.delete()) {
errors?.add("Failed to delete $label: ${file.absolutePath}")
}
} catch (e: Exception) {
errors?.add("Failed to delete $label: ${file.absolutePath} (${e.message})")
}
}
private fun scanLocalDirectory(path: String): List<DiscoveredFile> {
val dir = File(path)
if (!dir.exists() || !dir.isDirectory) return emptyList()
return dir.listFiles { f ->
f.isFile && (f.name.endsWith(".dsl") || f.name.endsWith(".dsl.dz") || f.name.endsWith(".dsl.gz"))
}?.map { file ->
return dir.listFiles { file -> isDictionaryPayloadFile(file) }?.map { file ->
DiscoveredFile(
name = file.name,
localPath = file.absolutePath,
@@ -351,10 +396,17 @@ class LocalDictionaryRepository(
} ?: emptyList()
}
private fun isDictionaryPayloadFile(file: File): Boolean =
file.isFile && (
file.name.endsWith(".dsl") ||
file.name.endsWith(".dsl.dz") ||
file.name.endsWith(".dsl.gz")
)
suspend fun search(query: String): OperationResult<List<IndexEntry>> = withContext(defaultDispatcher) {
if (query.isBlank()) return@withContext OperationResult.Success(emptyList())
val activeDicts = mutableDictionaries.value
val activeDicts = mutableDictionaries.value.filter { it.isActive }
val sanitizedQuery = query.sanitizeQuery()
ReSearchTrace.asyncSection(ReSearchTrace.SEARCH_DIRECT) {
@@ -404,65 +456,81 @@ class LocalDictionaryRepository(
return engine.getIndexSearcher()
}
fun toggleDictionaryActive(dictionaryPath: String) {
mutableDictionaries.update { currentList ->
currentList.map { dict ->
if (dict.path == dictionaryPath) {
dict.withActiveStatus(!dict.isActive)
} else {
dict
suspend fun toggleDictionaryActive(dictionaryPath: String) {
dictionaryStateMutex.withLock {
val isActive = mutableDictionaries.value
.firstOrNull { it.path == dictionaryPath }
?.isActive
?.not()
?: return
preferencesManager.setDictionaryActive(dictionaryPath, isActive)
mutableDictionaries.update { currentList ->
currentList.map { dict ->
if (dict.path == dictionaryPath) {
dict.withActiveStatus(isActive)
} else {
dict
}
}
}
}
}
fun deleteDictionary(dictionary: Dictionary): OperationResult<Unit> {
try {
mutableDictionaries.update { current ->
current.filter { it.path != dictionary.path }
}
private fun removeDictionaryFromState(path: String) {
mutableDictionaries.update { current -> current.filter { it.path != path } }
}
launch(ioDispatcher) {
try {
val deleteErrors = mutableListOf<String>()
val dictFile = File(dictionary.path)
if (dictFile.exists()) {
if (!dictFile.delete()) {
deleteErrors.add("Failed to delete dictionary file: ${dictFile.absolutePath}")
}
}
val indexFile = if (dictionary.indexPath.endsWith(".idx", ignoreCase = true)) {
File(dictionary.indexPath)
} else {
File("${dictionary.indexPath}.idx")
}
if (indexFile.exists()) {
if (!indexFile.delete()) {
deleteErrors.add("Failed to delete index file: ${indexFile.absolutePath}")
}
}
try {
engine.getIndexSearcher().trimMemory(80)
} catch (e: Exception) {
Log.w(TAG, "Failed to trim memory after deletion: ${e.message}")
}
if (deleteErrors.isNotEmpty()) {
val message = deleteErrors.joinToString("; ")
Log.w(TAG, "Dictionary deletion completed with errors: $message")
}
} catch (e: Exception) {
Log.e(TAG, "Background deletion failed: ${e.message}", e)
suspend fun deleteDictionary(dictionary: Dictionary): OperationResult<Unit> {
return try {
val (failure, cleanupErrors) = withContext(ioDispatcher) {
val dictFile = File(dictionary.path)
if (dictFile.exists() && !dictFile.delete()) {
return@withContext Pair(
"Failed to delete dictionary file: ${dictFile.absolutePath}",
emptyList(),
)
}
val deleteErrors = mutableListOf<String>()
val indexFile = if (dictionary.indexPath.endsWith(".idx", ignoreCase = true)) {
File(dictionary.indexPath)
} else {
File("${dictionary.indexPath}.idx")
}
deleteIndexFiles(indexFile, deleteErrors)
Pair<String?, List<String>>(null, deleteErrors)
}
return OperationResult.Success(Unit)
if (failure != null) {
return OperationResult.Error(failure)
}
dictionaryStateMutex.withLock {
try {
preferencesManager.removeDictionaryActiveState(dictionary.path)
} catch (e: Exception) {
Log.w(TAG, "Failed to clear dictionary active state: ${e.message}")
}
removeDictionaryFromState(dictionary.path)
}
try {
engine.getIndexSearcher().trimMemory(80)
} catch (e: Exception) {
Log.w(TAG, "Failed to trim memory after deletion: ${e.message}")
}
if (cleanupErrors.isNotEmpty()) {
Log.w(
TAG,
"Dictionary deletion completed with errors: ${cleanupErrors.joinToString("; ")}"
)
}
OperationResult.Success(Unit)
} catch (e: Exception) {
return OperationResult.Error("Failed to delete dictionary", e)
OperationResult.Error("Failed to delete dictionary", e)
}
}
@@ -507,13 +575,22 @@ class LocalDictionaryRepository(
if (count < 0) return null
var articleCount = count
if (version >= 6) {
if (indexFile.length() < INDEX_V6_HEADER_BYTES) return null
articleCount = dataInput.readInt()
if (articleCount < 0) return null
}
if (version >= 2) {
if (indexFile.length() < INDEX_V2_HEADER_BYTES) return null
val fixedHeaderBytes =
if (version >= 6) INDEX_V6_HEADER_BYTES else INDEX_V2_HEADER_BYTES
if (indexFile.length() < fixedHeaderBytes) return null
val sparseCount = dataInput.readInt()
if (sparseCount < 0) return null
var headerBytes = INDEX_V2_HEADER_BYTES
var headerBytes = fixedHeaderBytes
repeat(sparseCount) {
dataInput.readInt()
dataInput.readLong()
@@ -524,7 +601,7 @@ class LocalDictionaryRepository(
}
}
IndexHeader(version, count)
IndexHeader(version, articleCount)
}
}
} catch (e: Exception) {
@@ -602,6 +679,7 @@ class LocalDictionaryRepository(
const val MIN_PROGRESS_EMIT_INTERVAL_MS = 200L
const val LEGACY_INDEX_HEADER_BYTES = 8L
const val INDEX_V2_HEADER_BYTES = 12
const val INDEX_V6_HEADER_BYTES = 16
const val SPARSE_ENTRY_FIXED_BYTES = 14
}
}
@@ -12,6 +12,7 @@ import kotlin.time.Duration.Companion.seconds
class DownloadManager(
private val dictionaryRepository: DictionaryRepository,
private val unknownErrorMessage: String,
private val sourceUnavailableMessage: String,
dispatcher: CoroutineDispatcher = Dispatchers.IO,
externalScope: CoroutineScope? = null
) {
@@ -24,6 +25,7 @@ class DownloadManager(
mutableDownloadProgressState.asStateFlow()
private val mutex = Mutex()
private var downloadJob: Job? = null
private var cancelCleanupJob: Job? = null
companion object {
private const val TAG = "DownloadManager"
@@ -39,30 +41,51 @@ class DownloadManager(
val clampedProgress = progress.coerceIn(0f, 1f)
mutableDownloadProgressState.value = DownloadProgressState(state, clampedProgress)
mutableDownloadState.value = state
if (state is DownloadState.Success) {
onDownloadSuccess?.invoke()
} else if (state is DownloadState.Error || state is DownloadState.Cancelled) {
onTerminal?.invoke(state)
}
}
var onFlowStarted: (() -> Unit)? = null
var onDownloadSuccess: (() -> Unit)? = null
var onTerminal: ((DownloadState) -> Unit)? = null
fun startDownload() {
if (downloadState.value is DownloadState.Loading) return
if (!downloadScope.isActive) return
onFlowStarted?.invoke()
cancelCleanupJob = null
downloadJob = downloadScope.launch {
mutex.withLock {
try {
updateDownloadProgressState(DownloadState.Loading, 0f)
val hasEnabledSources = try {
dictionaryRepository.hasEnabledSources()
val enabledSources = try {
dictionaryRepository.getEnabledSources()
} catch (_: Exception) {
false
emptyList()
}
if (!hasEnabledSources) {
if (enabledSources.isEmpty()) {
dictionaryRepository.performAllCleanup()
updateDownloadProgressState(DownloadState.Success, 1f)
updateDownloadProgressState(DownloadState.Idle, 0f)
return@withLock
}
val downloadableSources =
dictionaryRepository.filterServerAvailableSources(enabledSources)
if (downloadableSources.isEmpty()) {
updateDownloadProgressState(
DownloadState.Error(sourceUnavailableMessage), 0f
)
return@withLock
}
updateDownloadProgressState(DownloadState.Loading, 0f)
var lastReportedProgressBucket = -1
val result = ReSearchTrace.asyncSection(ReSearchTrace.DICTIONARY_DOWNLOAD) {
dictionaryRepository.downloadDictionaries { progress ->
dictionaryRepository.downloadSources(downloadableSources) { progress ->
val progressBucket = progressBucket(progress, DOWNLOAD_PROGRESS_STEP_PERCENT)
if (progressBucket != lastReportedProgressBucket) {
lastReportedProgressBucket = progressBucket
@@ -94,7 +117,7 @@ class DownloadManager(
} catch (e: Exception) {
if (e is CancellationException) {
updateDownloadProgressState(DownloadState.Cancelled, 0f)
downloadScope.launch {
cancelCleanupJob = downloadScope.launch {
try {
dictionaryRepository.deleteUnprocessedFiles()
} catch (cleanupException: Exception) {
@@ -124,19 +147,41 @@ class DownloadManager(
fun startDownloadForSources(sourceUrls: List<String>) {
if (downloadState.value is DownloadState.Loading) return
if (!downloadScope.isActive) return
onFlowStarted?.invoke()
cancelCleanupJob = null
downloadJob = downloadScope.launch {
mutex.withLock {
try {
updateDownloadProgressState(DownloadState.Loading, 0f)
if (sourceUrls.isEmpty()) {
updateDownloadProgressState(DownloadState.Success, 1f)
updateDownloadProgressState(DownloadState.Idle, 0f)
return@withLock
}
val sources = try {
dictionaryRepository.getEnabledSources(sourceUrls)
} catch (_: Exception) {
emptyList()
}
if (sources.isEmpty()) {
updateDownloadProgressState(DownloadState.Idle, 0f)
return@withLock
}
val downloadableSources =
dictionaryRepository.filterServerAvailableSources(sources)
if (downloadableSources.isEmpty()) {
updateDownloadProgressState(
DownloadState.Error(sourceUnavailableMessage), 0f
)
return@withLock
}
updateDownloadProgressState(DownloadState.Loading, 0f)
var lastReportedProgressBucket = -1
val result = ReSearchTrace.asyncSection(ReSearchTrace.DICTIONARY_DOWNLOAD) {
dictionaryRepository.downloadSpecificSources(sourceUrls) { progress ->
dictionaryRepository.downloadSources(downloadableSources) { progress ->
val progressBucket = progressBucket(progress, DOWNLOAD_PROGRESS_STEP_PERCENT)
if (progressBucket != lastReportedProgressBucket) {
lastReportedProgressBucket = progressBucket
@@ -168,7 +213,7 @@ class DownloadManager(
} catch (e: Exception) {
if (e is CancellationException) {
updateDownloadProgressState(DownloadState.Cancelled, 0f)
downloadScope.launch {
cancelCleanupJob = downloadScope.launch {
try {
dictionaryRepository.deleteUnprocessedFiles()
} catch (cleanupException: Exception) {
@@ -195,10 +240,12 @@ class DownloadManager(
}
}
fun cancelDownload() {
downloadJob?.cancel()
val jobToCancel = downloadJob
jobToCancel?.cancel()
updateDownloadProgressState(DownloadState.Cancelled, 0f)
downloadScope.launch {
delay(TERMINAL_STATE_DURATION)
jobToCancel?.join()
cancelCleanupJob?.join()
if (downloadState.value == DownloadState.Cancelled) {
updateDownloadProgressState(DownloadState.Idle, 0f)
}
@@ -1,4 +1,8 @@
package com.example.research.feature.download.model
import androidx.compose.runtime.Immutable
@Immutable
sealed class DownloadState {
data object Idle : DownloadState()
data object Loading : DownloadState()
@@ -7,3 +11,6 @@ sealed class DownloadState {
data class Error(val message: String) : DownloadState()
data object Cancelled : DownloadState()
}
val DownloadState.isActive: Boolean
get() = this is DownloadState.Loading || this is DownloadState.Extracting || this is DownloadState.Success
@@ -12,6 +12,12 @@ import kotlinx.coroutines.withContext
import okhttp3.OkHttpClient
import okhttp3.Request
enum class SourceAvailability {
AVAILABLE,
NOT_FOUND,
UNKNOWN
}
class DictionaryChecker(
private val fileStorageManager: FileStorageManager,
private val client: OkHttpClient
@@ -85,7 +91,26 @@ class DictionaryChecker(
}
}
private fun isSourceAvailableOnServer(source: DictionarySource, date: String): Boolean {
suspend fun checkSourceAvailability(source: DictionarySource): SourceAvailability =
withContext(Dispatchers.IO) {
val currentDate = DateUtils.getCurrentDateString()
when (probeSource(source, currentDate)) {
SourceAvailability.AVAILABLE -> SourceAvailability.AVAILABLE
SourceAvailability.UNKNOWN -> SourceAvailability.UNKNOWN
SourceAvailability.NOT_FOUND -> {
if (DictionarySource.hasDatePlaceholder(source.urlTemplate)) {
probeSource(source, DateUtils.getPreviousDateString())
} else {
SourceAvailability.NOT_FOUND
}
}
}
}
private fun isSourceAvailableOnServer(source: DictionarySource, date: String): Boolean =
probeSource(source, date) == SourceAvailability.AVAILABLE
private fun probeSource(source: DictionarySource, date: String): SourceAvailability {
return try {
val url = DictionarySource.buildUrl(source.urlTemplate, date)
val request = Request.Builder()
@@ -94,12 +119,17 @@ class DictionaryChecker(
.build()
client.newCall(request).execute().use { response ->
response.isSuccessful &&
if (response.isSuccessful &&
response.header("Content-Type")?.startsWith("text/html") != true
) {
SourceAvailability.AVAILABLE
} else {
SourceAvailability.NOT_FOUND
}
}
} catch (e: Exception) {
Log.w(TAG, "Failed to check source availability for ${source.urlTemplate}: ${e.message}")
false
SourceAvailability.UNKNOWN
}
}
@@ -59,11 +59,11 @@ class DictionaryCleaner(
toDelete.add(item.name)
continue
}
if ((item.ext == "idx" || item.ext == "dsl.idx") && item.date != latestDsl) {
if (item.ext in indexExtensions && item.date != latestDsl) {
toDelete.add(item.name)
continue
}
if ((item.ext == "idx" || item.ext == "dsl.idx") && item.date !in dslDates) {
if (item.ext in indexExtensions && item.date !in dslDates) {
toDelete.add(item.name)
}
}
@@ -76,10 +76,13 @@ class DictionaryCleaner(
val date: String,
val ext: String,
)
private val validExtensions = setOf("dsl", "dsl.dz", "dsl.gz", "gz", "idx", "dsl.idx")
private val indexExtensions = setOf("idx", "dsl.idx", "idx.fold", "dsl.idx.fold")
private val validExtensions = setOf("dsl", "dsl.dz", "dsl.gz", "gz") + indexExtensions
private fun parseManagedName(name: String, prefixes: Set<String>): ManagedFileName? {
val ext = when {
name.endsWith(".dsl.idx.fold") -> "dsl.idx.fold"
name.endsWith(".idx.fold") -> "idx.fold"
name.endsWith(".dsl.dz") -> "dsl.dz"
name.endsWith(".dsl.gz") -> "dsl.gz"
name.endsWith(".dsl.idx") -> "dsl.idx"
@@ -343,6 +343,8 @@ class DictionaryDownloader(
context.getString(R.string.download_connection_reset)
e is SecurityException ->
context.getString(R.string.download_no_write_permission)
e.isNetworkError() ->
context.getString(R.string.download_network_failed)
else ->
context.getString(R.string.download_error)
}
@@ -8,6 +8,8 @@ import com.example.research.core.domain.model.DictionarySource
import com.example.research.data.local.preferences.PreferencesManager
import com.example.research.common.util.FileStorageManager
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.withContext
import okhttp3.OkHttpClient
@@ -40,74 +42,58 @@ class DictionaryRepository(
checker.areDictionariesUpToDate(uri, sourcesToCheck, files)
}
suspend fun hasEnabledSources(): Boolean = withContext(Dispatchers.IO) {
preferencesManager.dictionarySources.first().any { it.isEnabled }
}
suspend fun downloadDictionaries(
onProgress: (Float) -> Unit
): Result<Unit> = withContext(Dispatchers.IO) {
val folderUri = preferencesManager.dictionaryPath
val uri = Uri.fromFile(File(folderUri))
val files = fileStorageManager.listFilesInFolder(uri)
cleaner.deleteCorruptedArchives(uri, files)
synchronized(filesBeforeDownload) {
filesBeforeDownload = files.map { it.name }.toMutableSet()
}
val sources = preferencesManager.dictionarySources.first()
val enabledSources = sources.filter { it.isEnabled }
synchronized(downloadedPrefixes) {
downloadedPrefixes = enabledSources.mapNotNull {
DictionarySource.extractPrefix(it.urlTemplate)
}.toMutableSet()
}
downloader.downloadDictionaries(
folderUri = uri,
sources = enabledSources,
onProgress = onProgress
)
}
suspend fun downloadSpecificSources(
sourceUrls: List<String>,
onProgress: (Float) -> Unit
): Result<Unit> = withContext(Dispatchers.IO) {
val folderUri = preferencesManager.dictionaryPath
val uri = Uri.fromFile(File(folderUri))
val files = fileStorageManager.listFilesInFolder(uri)
cleaner.deleteCorruptedArchives(uri, files)
synchronized(filesBeforeDownload) {
filesBeforeDownload = files.map { it.name }.toMutableSet()
}
val allSources = preferencesManager.dictionarySources.first()
val sourceKeys = sourceUrls.mapTo(mutableSetOf(), DictionarySource::identityKey)
val sourcesToDownload = allSources.mapNotNull { source ->
if (!source.isEnabled || DictionarySource.identityKey(source.urlTemplate) !in sourceKeys) {
null
} else {
source.copy(urlTemplate = DictionarySource.normalizeTemplate(source.urlTemplate))
suspend fun getEnabledSources(sourceUrls: List<String>? = null): List<DictionarySource> =
withContext(Dispatchers.IO) {
val enabledSources = preferencesManager.dictionarySources.first()
.filter { it.isEnabled }
if (sourceUrls == null) {
return@withContext enabledSources
}
val sourceKeys = sourceUrls.mapTo(mutableSetOf(), DictionarySource::identityKey)
enabledSources.mapNotNull { source ->
if (DictionarySource.identityKey(source.urlTemplate) !in sourceKeys) {
null
} else {
source.copy(urlTemplate = DictionarySource.normalizeTemplate(source.urlTemplate))
}
}
}
if (sourcesToDownload.isEmpty()) {
return@withContext Result.success(Unit)
suspend fun filterServerAvailableSources(
sources: List<DictionarySource>
): List<DictionarySource> = withContext(Dispatchers.IO) {
if (sources.isEmpty()) {
return@withContext emptyList()
}
sources
.map { source -> async { source to checker.checkSourceAvailability(source) } }
.awaitAll()
.filter { (_, availability) -> availability != SourceAvailability.NOT_FOUND }
.map { (source, _) -> source }
}
suspend fun downloadSources(
sources: List<DictionarySource>,
onProgress: (Float) -> Unit
): Result<Unit> = withContext(Dispatchers.IO) {
val folderUri = preferencesManager.dictionaryPath
val uri = Uri.fromFile(File(folderUri))
val files = fileStorageManager.listFilesInFolder(uri)
cleaner.deleteCorruptedArchives(uri, files)
synchronized(filesBeforeDownload) {
filesBeforeDownload = files.map { it.name }.toMutableSet()
}
synchronized(downloadedPrefixes) {
downloadedPrefixes = sourcesToDownload.mapNotNull {
downloadedPrefixes = sources.mapNotNull {
DictionarySource.extractPrefix(it.urlTemplate)
}.toMutableSet()
}
downloader.downloadDictionaries(
folderUri = uri,
sources = sourcesToDownload,
sources = sources,
onProgress = onProgress
)
}
@@ -3,19 +3,19 @@ package com.example.research.feature.download.service
import android.app.Service
import android.content.Intent
import android.os.IBinder
import android.util.Log
import androidx.core.app.ServiceCompat
import com.example.research.ReSearchApplication
import com.example.research.common.progress.renderTitle
import com.example.research.common.progress.DictionaryPipelineCoordinator
import com.example.research.common.progress.DictionaryProgressPresenter
import com.example.research.common.util.NotificationHelper
import com.example.research.core.util.OperationResult
import com.example.research.data.repository.LocalDictionaryRepository
import com.example.research.feature.download.DownloadManager
import com.example.research.feature.download.model.DownloadState
import kotlinx.coroutines.*
import kotlinx.coroutines.flow.*
import kotlin.time.Duration.Companion.milliseconds
import kotlin.time.Duration.Companion.seconds
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.cancelChildren
import kotlinx.coroutines.Job
class DictionaryForegroundService : Service() {
@@ -23,21 +23,21 @@ class DictionaryForegroundService : Service() {
const val ACTION_START = "com.example.research.START_DOWNLOAD"
const val ACTION_STOP = "com.example.research.STOP_DOWNLOAD"
const val ACTION_IMPORT = "com.example.research.START_IMPORT"
const val ACTION_FINISH = "com.example.research.FINISH_PIPELINE"
private const val NOTIFICATION_ID = NotificationHelper.NOTIFICATION_ID
}
private lateinit var downloadManager: DownloadManager
private lateinit var localDictionaryRepository: LocalDictionaryRepository
private lateinit var dictionaryImportManager: com.example.research.feature.import.DictionaryImportManager
private lateinit var progressStateHolder: com.example.research.common.progress.DictionaryProgressStateHolder
private lateinit var pipelineCoordinator: DictionaryPipelineCoordinator
private lateinit var presenter: DictionaryProgressPresenter
private val notificationHelper by lazy {
NotificationHelper(this)
}
private val serviceScope = CoroutineScope(SupervisorJob() + Dispatchers.Main)
@Volatile
private var isHandlingSuccess = false
@Volatile
private var latestStartId = 0
override fun onCreate() {
@@ -47,164 +47,8 @@ class DictionaryForegroundService : Service() {
downloadManager = app.downloadManager
localDictionaryRepository = app.localDictionaryRepository
dictionaryImportManager = app.dictionaryImportManager
progressStateHolder = app.dictionaryProgressStateHolder
observeDownloadState()
observeImportState()
observeUnifiedProgress()
}
private fun observeDownloadState() {
serviceScope.launch {
downloadManager.downloadState.collect { state ->
when (state) {
is DownloadState.Success -> handleSuccess()
is DownloadState.Error -> {
notificationHelper.showErrorNotification()
stopForegroundService(removeNotification = false)
}
is DownloadState.Cancelled -> {
notificationHelper.cancelNotification()
stopForegroundService()
}
else -> {}
}
}
}
}
private fun observeImportState() {
serviceScope.launch {
dictionaryImportManager.importState.collect { state ->
when (state) {
is com.example.research.ui.settings.ImportState.Success -> handleSuccess()
is com.example.research.ui.settings.ImportState.Error -> {
notificationHelper.showErrorNotification()
stopForegroundService(removeNotification = false)
}
else -> {}
}
}
}
}
private fun observeUnifiedProgress() {
serviceScope.launch {
progressStateHolder.progressSnapshot
.collect { snapshot ->
if (snapshot != null) {
notificationHelper.showUnifiedProgressNotification(
title = snapshot.renderTitle(this@DictionaryForegroundService),
contentText = "${snapshot.percent}%",
progressPercent = snapshot.percent,
)
}
}
}
}
private suspend fun triggerReindexing(isImportFlow: Boolean) = withContext(Dispatchers.IO) {
try {
waitForIndexingCompletion()
delay(1.seconds)
val app = application as? ReSearchApplication
?: return@withContext
val path = app.preferencesManager.dictionaryPath
if (isImportFlow) {
dictionaryImportManager.updateExtractionProgress(0f)
}
app.downloadDictionaryRepository.extractArchives(onProgress = { progress ->
if (isImportFlow) {
dictionaryImportManager.updateExtractionProgress(progress)
}
})
val result = localDictionaryRepository.scanDirectory(path)
if (result is OperationResult.Success && result.data > 0) {
localDictionaryRepository.warmupIndexes()
app.downloadDictionaryRepository.performAllCleanup()
}
if (isImportFlow) {
dictionaryImportManager.markImportPipelineSuccess()
}
} catch (e: CancellationException) {
throw e
} catch (e: Exception) {
Log.e("DictionaryForegroundService", "triggerReindexing() failed: ${e.message}", e)
}
}
private suspend fun waitForIndexingCompletion(): Boolean {
val isIndexing = localDictionaryRepository.indexingProgress.first().isIndexing
if (!isIndexing) {
return true
}
var waitCount = 0
while (localDictionaryRepository.indexingProgress.first().isIndexing && waitCount < 100) {
delay(200.milliseconds)
waitCount++
}
return waitCount < 100
}
private fun handleSuccess() {
if (isHandlingSuccess) return
isHandlingSuccess = true
serviceScope.launch {
try {
val isImportFlow = dictionaryImportManager.importState.value !is com.example.research.ui.settings.ImportState.Idle
if (isImportFlow) {
val app = application as? ReSearchApplication
val path = app?.preferencesManager?.dictionaryPath
val dictionariesDir = path?.let { java.io.File(it) }
val filesBeforeReindex = dictionariesDir?.listFiles()?.map { it.name }?.toSet() ?: emptySet()
try {
triggerReindexing(true)
dictionaryImportManager.getAndClearImportedFiles()
} catch (e: CancellationException) {
val filesToCleanup = dictionaryImportManager.getAndClearImportedFiles()
filesToCleanup.forEach { file ->
try { if (file.exists()) file.delete() } catch (_: Exception) { /* Ignore cleanup failure */ }
}
if (dictionariesDir != null) {
cleanupNewFiles(dictionariesDir, filesBeforeReindex)
}
throw e
}
} else {
triggerReindexing(false)
}
if (isImportFlow) {
notificationHelper.showImportSuccessNotification()
dictionaryImportManager.clearImportState()
} else {
notificationHelper.showSuccessNotification()
}
} catch (e: Exception) {
if (e !is CancellationException) {
Log.e("DownloadForegroundService", "handleSuccess() failed: ${e.message}", e)
notificationHelper.showErrorNotification()
}
} finally {
isHandlingSuccess = false
stopForegroundService(removeNotification = false)
}
}
}
private fun cleanupNewFiles(dir: java.io.File, filesBeforeSnapshot: Set<String>) {
try {
dir.listFiles()?.forEach { file ->
if (file.name !in filesBeforeSnapshot) {
try { file.delete() } catch (_: Exception) { /* Ignore delete failure */ }
}
}
} catch (_: Exception) { /* Ignore directory listing failure */ }
pipelineCoordinator = app.dictionaryPipelineCoordinator
presenter = app.dictionaryProgressPresenter
}
private fun startForegroundService() {
@@ -249,15 +93,20 @@ class DictionaryForegroundService : Service() {
}
private fun stopForTimeout(startId: Int) {
cancelAllWork()
ServiceCompat.stopForeground(this, ServiceCompat.STOP_FOREGROUND_REMOVE)
stopSelfResult(startId)
}
private fun cancelAllWork() {
serviceScope.coroutineContext[Job]?.cancelChildren()
pipelineCoordinator.cancelPipeline()
if (localDictionaryRepository.isIndexingInProgress()) {
localDictionaryRepository.cancelIndexing()
}
downloadManager.cancelDownload()
dictionaryImportManager.cancelImport()
notificationHelper.cancelNotification()
ServiceCompat.stopForeground(this, ServiceCompat.STOP_FOREGROUND_REMOVE)
stopSelfResult(startId)
presenter.cancelProgress()
}
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
@@ -275,21 +124,16 @@ class DictionaryForegroundService : Service() {
ACTION_IMPORT -> {
startForegroundService()
}
ACTION_FINISH -> {
stopForegroundService(removeNotification = false)
}
ACTION_STOP -> {
try {
startForegroundService()
} catch (_: Exception) {
// Service may already be stopped
}
serviceScope.coroutineContext[Job]?.cancelChildren()
if (localDictionaryRepository.isIndexingInProgress()) {
localDictionaryRepository.cancelIndexing()
}
downloadManager.cancelDownload()
dictionaryImportManager.cancelImport()
notificationHelper.cancelNotification()
cancelAllWork()
stopForegroundService()
}
}
@@ -3,6 +3,7 @@ package com.example.research.feature.import
import android.app.Application
import android.net.Uri
import com.example.research.R
import com.example.research.common.progress.ImportFlowOperations
import com.example.research.common.util.SafeFileName
import com.example.research.core.performance.ReSearchTrace
import com.example.research.ui.settings.ImportState
@@ -14,16 +15,19 @@ import java.util.Collections
class DictionaryImportManager(
private val application: Application
) {
) : ImportFlowOperations {
private val mutableImportState = MutableStateFlow<ImportState>(ImportState.Idle)
val importState: StateFlow<ImportState> = mutableImportState.asStateFlow()
private val managerScope = CoroutineScope(SupervisorJob() + Dispatchers.Main)
private var importJob: Job? = null
private val importedFiles = Collections.synchronizedList(mutableListOf<File>())
var onFlowStarted: (() -> Unit)? = null
var onTerminal: ((ImportState) -> Unit)? = null
fun importDictionaries(uris: List<Uri>) {
importJob?.cancel()
onFlowStarted?.invoke()
importJob = managerScope.launch {
mutableImportState.value = ImportState.Idle
performImport(uris)
@@ -36,6 +40,8 @@ class DictionaryImportManager(
val dictionariesDir = File(context.getExternalFilesDir(null), "dictionaries")
val totalFiles = uris.size.coerceAtLeast(1)
importedFiles.clear()
val skippedNames = mutableListOf<String>()
val deferredErrors = mutableListOf<String>()
if (!dictionariesDir.exists()) {
dictionariesDir.mkdirs()
@@ -46,9 +52,7 @@ class DictionaryImportManager(
currentCoroutineContext().ensureActive()
val fileName = getFileName(uri) ?: continue
if (SafeFileName.validate(fileName) == null) {
mutableImportState.value = ImportState.Error(
context.getString(R.string.import_invalid_file_name, fileName)
)
deferredErrors += context.getString(R.string.import_invalid_file_name, fileName)
continue
}
val lowerFileName = fileName.lowercase()
@@ -65,9 +69,7 @@ class DictionaryImportManager(
val destFile = File(dictionariesDir, fileName)
if (destFile.exists()) {
mutableImportState.value = ImportState.Error(
context.getString(R.string.import_file_exists, fileName)
)
skippedNames += fileName
continue
}
@@ -115,16 +117,30 @@ class DictionaryImportManager(
currentCoroutineContext().ensureActive()
mutableImportState.value = ImportState.Success
val terminalState = resolveImportOutcome(
importedCount = importedFiles.size,
skippedNames = skippedNames,
deferredErrors = deferredErrors,
).toImportState(
existsMessage = { names ->
context.getString(R.string.import_file_exists, names.joinToString(separator = ", "))
},
invalidMessage = { message -> message },
nothingImportedMessage = { context.getString(R.string.import_nothing_imported) },
)
mutableImportState.value = terminalState
onTerminal?.invoke(terminalState)
} catch (e: CancellationException) {
cleanupImportedFiles()
mutableImportState.value = ImportState.Idle
throw e
} catch (e: Exception) {
mutableImportState.value = ImportState.Error(
val errorState = ImportState.Error(
context.getString(R.string.import_error, e.message ?: "Unknown error")
)
mutableImportState.value = errorState
onTerminal?.invoke(errorState)
}
}
@@ -154,18 +170,18 @@ class DictionaryImportManager(
return size
}
fun clearImportState() {
override fun clearImportState() {
mutableImportState.value = ImportState.Idle
}
fun updateExtractionProgress(progress: Float) {
override fun updateExtractionProgress(progress: Float) {
mutableImportState.update { current ->
if (current is ImportState.Idle || current is ImportState.Error) current
else ImportState.Extracting(progress.coerceIn(0f, 1f))
}
}
fun markImportPipelineSuccess() {
override fun markImportPipelineSuccess() {
mutableImportState.update { current ->
if (current is ImportState.Idle || current is ImportState.Error) current
else ImportState.Success
@@ -179,7 +195,7 @@ class DictionaryImportManager(
mutableImportState.value = ImportState.Idle
}
fun getAndClearImportedFiles(): List<File> {
override fun getAndClearImportedFiles(): List<File> {
val files = importedFiles.toList()
importedFiles.clear()
return files
@@ -0,0 +1,32 @@
package com.example.research.feature.import
import com.example.research.ui.settings.ImportState
internal sealed interface ImportOutcome {
data object Imported : ImportOutcome
data class AlreadyExists(val names: List<String>) : ImportOutcome
data class InvalidFile(val message: String) : ImportOutcome
data object NothingImported : ImportOutcome
}
internal fun resolveImportOutcome(
importedCount: Int,
skippedNames: List<String>,
deferredErrors: List<String>,
): ImportOutcome = when {
importedCount > 0 -> ImportOutcome.Imported
skippedNames.isNotEmpty() -> ImportOutcome.AlreadyExists(skippedNames)
deferredErrors.isNotEmpty() -> ImportOutcome.InvalidFile(deferredErrors.first())
else -> ImportOutcome.NothingImported
}
internal fun ImportOutcome.toImportState(
existsMessage: (List<String>) -> String,
invalidMessage: (String) -> String,
nothingImportedMessage: () -> String,
): ImportState = when (this) {
ImportOutcome.Imported -> ImportState.Success
is ImportOutcome.AlreadyExists -> ImportState.Error(existsMessage(names))
is ImportOutcome.InvalidFile -> ImportState.Error(invalidMessage(message))
ImportOutcome.NothingImported -> ImportState.Error(nothingImportedMessage())
}
@@ -85,7 +85,12 @@ class SearchViewModel(
@OptIn(FlowPreview::class, ExperimentalCoroutinesApi::class)
val searchResults: Flow<PagingData<IndexEntry>> = combine(
searchQuery.debounce(300.milliseconds).distinctUntilChanged(),
searchQuery
.map { it to it.isBlank() }
.distinctUntilChangedBy { it.first }
.flatMapLatest { (query, isBlank) ->
if (isBlank) flowOf(query) else flowOf(query).debounce(300.milliseconds)
},
activeDictionaries
) { query, dictionaries ->
query to dictionaries
@@ -98,8 +103,7 @@ class SearchViewModel(
pageSize = 12,
prefetchDistance = 1,
enablePlaceholders = false,
initialLoadSize = 12,
jumpThreshold = 1
initialLoadSize = 12
),
pagingSourceFactory = {
IndexEntryPagingSource(
@@ -78,6 +78,4 @@ internal object AboutLibrariesParser {
context.resources.openRawResource(resourceId)
.bufferedReader()
.use { reader -> json.decodeFromString(reader.readText()) }
fun decode(source: String): AboutLibrariesData = json.decodeFromString(source)
}
@@ -1,6 +1,5 @@
package com.example.research.ui.about
import androidx.compose.animation.animateContentSize
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ExperimentalLayoutApi
@@ -27,6 +26,7 @@ import androidx.compose.material3.TopAppBarDefaults
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.produceState
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
@@ -43,6 +43,8 @@ import androidx.compose.ui.unit.dp
import com.example.research.R
import com.example.research.common.ui.components.OutlinedChoiceButton
import com.example.research.ui.theme.AppWindowInsets
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
@OptIn(ExperimentalMaterial3Api::class, ExperimentalLayoutApi::class)
@Composable
@@ -57,8 +59,10 @@ fun AboutScreen(
}
val versionName = packageInfo?.versionName ?: stringResource(R.string.version_unknown)
val uriHandler = LocalUriHandler.current
val librariesData = remember(context) {
AboutLibrariesParser.read(context, R.raw.aboutlibraries)
val librariesData by produceState<AboutLibrariesData?>(initialValue = null, context) {
value = withContext(Dispatchers.IO) {
AboutLibrariesParser.read(context, R.raw.aboutlibraries)
}
}
var expandedLibraryId by remember { mutableStateOf<String?>(null) }
var dialogLicense by remember { mutableStateOf<AboutLicense?>(null) }
@@ -93,12 +97,12 @@ fun AboutScreen(
AboutHeader(versionName = versionName)
}
items(
items = librariesData.libraries,
items = librariesData?.libraries.orEmpty(),
key = AboutLibrary::uniqueId,
) { library ->
AboutLibraryRow(
library = library,
licenses = librariesData.licenses,
licenses = librariesData?.licenses.orEmpty(),
expanded = expandedLibraryId == library.uniqueId,
onToggle = {
expandedLibraryId = if (expandedLibraryId == library.uniqueId) {
@@ -191,8 +195,7 @@ private fun AboutLibraryRow(
) {
Column(
modifier = Modifier
.padding(horizontal = 16.dp, vertical = 12.dp)
.animateContentSize(),
.padding(horizontal = 16.dp, vertical = 12.dp),
verticalArrangement = Arrangement.spacedBy(5.dp)
) {
Row(
@@ -23,7 +23,6 @@ object DslAnnotatedParser {
"br", "p", "b", "i", "c", "t", "m", "m0", "m1", "m2", "m3", "m4", "m5",
"ref", "ex", "e", "trn", "com", "lang", "sup", "'"
)
private val ESCAPED_CHARS = setOf('[', ']', '(', ')')
data class ColorScheme(
val secondaryText: Color,
@@ -101,15 +100,13 @@ object DslAnnotatedParser {
'\\' -> {
if (i + 1 < length) {
val next = dsl[i + 1]
if (next in ESCAPED_CHARS) {
builder.append(next)
if (refStack.isNotEmpty()) {
refStack.last().second.append(next)
}
lastChar = next
i += 2
continue
builder.append(next)
if (refStack.isNotEmpty()) {
refStack.last().second.append(next)
}
lastChar = next
i += 2
continue
}
builder.append(char)
if (refStack.isNotEmpty()) {
@@ -119,7 +116,7 @@ object DslAnnotatedParser {
i++
}
'[' -> {
val end = dsl.indexOf(']', i + 1)
val end = tagCloseIndex(dsl, i + 1)
if (end != -1) {
val tagStart = i + 1
var tagEnd = end
@@ -228,6 +225,19 @@ object DslAnnotatedParser {
splitOversizedBlocks(result)
}
private fun tagCloseIndex(value: String, startIndex: Int): Int {
var index = startIndex
while (index < value.length) {
when {
value[index] == '\\' && index + 1 < value.length -> index += 2
value[index] == '[' -> return -1
value[index] == ']' -> return index
else -> index++
}
}
return -1
}
private fun splitOversizedBlocks(blocks: List<DslBlock>): List<DslBlock> {
if (blocks.none { it.text.length > MAX_BLOCK_TEXT_LENGTH }) return blocks
return blocks.flatMap { block ->
@@ -4,6 +4,7 @@ import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.offset
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.windowInsetsPadding
@@ -135,7 +136,9 @@ private fun SearchField(
Icon(
painter = searchIcon,
contentDescription = null,
modifier = Modifier.size(24.dp)
modifier = Modifier
.offset(x = 4.dp)
.size(24.dp)
)
},
trailingIcon = if (searchState.text.isNotEmpty()) {
@@ -146,6 +149,7 @@ private fun SearchField(
onClearQuery()
},
modifier = Modifier
.padding(end = 12.dp)
.testTag("clear_search")
.semantics { testTagsAsResourceId = true }
) {
@@ -41,7 +41,7 @@ fun SearchResultsList(
) {
items(
count = results.itemCount,
key = results.itemKey { entry -> "${entry.dictionaryPath}_${entry.offset.value}_${entry.word}" },
key = results.itemKey { "${it.dictionaryPath}:${it.offset.value}:${it.word}" },
contentType = results.itemContentType { "search_result" }
) { index ->
results[index]?.let { entry ->
@@ -2,22 +2,18 @@ package com.example.research.ui.navigation
import androidx.activity.compose.BackHandler
import androidx.compose.material3.adaptive.ExperimentalMaterial3AdaptiveApi
import androidx.compose.material3.adaptive.currentWindowAdaptiveInfo
import androidx.compose.material3.adaptive.currentWindowAdaptiveInfoV2
import androidx.compose.runtime.*
import androidx.compose.runtime.saveable.*
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.window.core.layout.WindowSizeClass.Companion.WIDTH_DP_MEDIUM_LOWER_BOUND
import com.example.research.feature.search.SearchAction
import com.example.research.feature.search.SearchViewModel
import com.example.research.feature.download.model.DownloadState
import com.example.research.ui.about.AboutScreen
import com.example.research.ui.article.ArticleRoute
import com.example.research.ui.main.MainRoute
import com.example.research.ui.settings.ImportState
import com.example.research.ui.settings.SettingsRoute
import com.example.research.ui.settings.SettingsViewModel
import kotlinx.coroutines.delay
import kotlin.time.Duration.Companion.milliseconds
private data class NavigationState(
val shouldShowSettings: Boolean,
@@ -33,7 +29,8 @@ enum class Screen {
@Composable
fun AppNavigation(
searchViewModel: SearchViewModel,
settingsViewModel: SettingsViewModel
settingsViewModel: SettingsViewModel,
seedNoDictionaries: Boolean = false
) {
val screenStack = rememberSaveable(
saver = listSaver(
@@ -53,39 +50,15 @@ fun AppNavigation(
val currentScreen = screenStack.lastOrNull() ?: Screen.Home
val settingsState by settingsViewModel.uiState.collectAsStateWithLifecycle()
val progressSnapshot by settingsViewModel.progressSnapshot.collectAsStateWithLifecycle()
val isPipelineActiveByState = remember(
progressSnapshot,
settingsState.indexingProgress.isIndexing,
settingsState.downloadState,
settingsState.importState,
) {
progressSnapshot != null ||
settingsState.indexingProgress.isIndexing ||
settingsState.downloadState is DownloadState.Loading ||
settingsState.downloadState is DownloadState.Extracting ||
settingsState.downloadState is DownloadState.Success ||
settingsState.importState is ImportState.Importing ||
settingsState.importState is ImportState.Extracting ||
settingsState.importState is ImportState.Success
}
var stickyPipelineActive by rememberSaveable { mutableStateOf(false) }
LaunchedEffect(isPipelineActiveByState) {
if (isPipelineActiveByState) {
stickyPipelineActive = true
} else {
delay(1500.milliseconds)
stickyPipelineActive = false
}
}
val navState by remember {
derivedStateOf {
val isOperationActive = stickyPipelineActive
val isOperationActive = !settingsState.pipelineIdle
val hasConfirmedNoDictionaries =
settingsState.hasCompletedStartupScan && settingsState.dictionaries.isEmpty()
val hasSeededNoDictionaries = seedNoDictionaries && !settingsState.hasCompletedStartupScan
NavigationState(
shouldShowSettings = hasConfirmedNoDictionaries || isOperationActive,
shouldShowSettings = hasConfirmedNoDictionaries || hasSeededNoDictionaries || isOperationActive,
showBackButtonInSettings = settingsState.dictionaries.isNotEmpty() && !isOperationActive,
isOperationActive = isOperationActive,
)
@@ -95,7 +68,7 @@ fun AppNavigation(
val shouldShowSettings = navState.shouldShowSettings
val showBackButtonInSettings = navState.showBackButtonInSettings
val isOperationActive = navState.isOperationActive
val isWideScreen = currentWindowAdaptiveInfo()
val isWideScreen = currentWindowAdaptiveInfoV2()
.windowSizeClass
.isWidthAtLeastBreakpoint(WIDTH_DP_MEDIUM_LOWER_BOUND)
@@ -1,5 +1,8 @@
package com.example.research.ui.settings
import androidx.compose.runtime.Immutable
@Immutable
sealed interface ImportState {
data object Idle : ImportState
data class Importing(val progress: Float) : ImportState
@@ -7,3 +10,6 @@ sealed interface ImportState {
data object Success : ImportState
data class Error(val message: String) : ImportState
}
val ImportState.isActive: Boolean
get() = this is ImportState.Importing || this is ImportState.Extracting || this is ImportState.Success
@@ -1,6 +1,7 @@
package com.example.research.ui.settings
import android.net.Uri
import androidx.compose.runtime.Immutable
import com.example.research.core.domain.model.AppTheme
import com.example.research.core.domain.model.Dictionary
import com.example.research.core.domain.model.DictionarySource
@@ -8,6 +9,7 @@ import com.example.research.core.domain.model.IndexingProgress
import com.example.research.DictionaryStatus
import com.example.research.feature.download.model.DownloadState
@Immutable
data class SettingsUiState(
val theme: AppTheme = AppTheme.SYSTEM,
val language: String = "system",
@@ -21,6 +23,7 @@ data class SettingsUiState(
val importState: ImportState = ImportState.Idle,
val dictionarySources: List<DictionarySource> = emptyList(),
val hasCompletedStartupScan: Boolean = false,
val pipelineIdle: Boolean = true,
val appVersion: String = "1.0",
)
@@ -11,6 +11,7 @@ import com.example.research.core.domain.model.AppTheme
import com.example.research.core.domain.model.Dictionary
import com.example.research.core.domain.model.DictionarySource
import com.example.research.core.domain.model.IndexingProgress
import com.example.research.core.domain.usecase.DictionarySourceFileMatcher
import com.example.research.core.domain.usecase.DictionarySourceValidator
import com.example.research.core.domain.usecase.ManageDictionarySourcesUseCase
import com.example.research.core.util.OperationResult
@@ -18,6 +19,7 @@ import com.example.research.data.local.preferences.PreferencesManager
import com.example.research.data.repository.LocalDictionaryRepository
import com.example.research.feature.download.DownloadManager
import com.example.research.feature.download.model.DownloadState
import com.example.research.feature.download.model.isActive
import com.example.research.feature.download.repository.DictionaryRepository
import com.example.research.feature.import.DictionaryImportManager
import kotlinx.coroutines.Dispatchers
@@ -30,7 +32,7 @@ import kotlinx.coroutines.flow.map
import kotlinx.coroutines.flow.receiveAsFlow
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
import java.io.File
import java.util.concurrent.ConcurrentHashMap
private data class DictionaryStateInputs(
val theme: AppTheme,
@@ -79,16 +81,29 @@ class SettingsViewModel(
private val language = MutableStateFlow("system")
private val hasCompletedStartupScan = MutableStateFlow(false)
private val pendingSourceUrls = mutableSetOf<String>()
private val pendingSourceUrls = ConcurrentHashMap.newKeySet<String>()
private var isDownloadInProgress = false
private var cancelRefreshPending = false
private var errorStateHandled = false
private var statusBeforeDownload: DictionaryStatus = DictionaryStatus.Unknown
init {
setupStateObservation()
observeDictionariesForSeedFlag()
initialize()
}
private fun observeDictionariesForSeedFlag() {
viewModelScope.launch {
localDictionaryRepository.dictionaries.collect { dictionaries ->
if (hasCompletedStartupScan.value) {
preferencesManager.setHadNoDictionaries(dictionaries.isEmpty())
}
}
}
}
private fun setupStateObservation() {
viewModelScope.launch {
var wasIndexing = false
@@ -146,37 +161,37 @@ class SettingsViewModel(
is DownloadState.Loading, is DownloadState.Extracting -> {
isDownloadInProgress = true
cancelRefreshPending = false
errorStateHandled = false
}
is DownloadState.Success -> {
pendingSourceUrls.clear()
}
is DownloadState.Error -> {
isDownloadInProgress = false
dictionaryStatus.value = if (dictionaryState.dictionaries.isEmpty()) {
DictionaryStatus.Empty
} else {
DictionaryStatus.UpToDate
}
if (pendingSourceUrls.isNotEmpty()) {
pendingSourceUrls.forEach { urlTemplate ->
val source = dictionaryState.dictionarySources.find {
it.urlTemplate == urlTemplate
}
if (source != null) {
val hasDictionary = dictionaryState.dictionaries.any { dict ->
DictionarySource.matchesDictionaryFile(
urlTemplate,
File(dict.path).name
)
if (!errorStateHandled) {
errorStateHandled = true
dictionaryStatus.value = if (dictionaryState.dictionaries.isEmpty()) {
DictionaryStatus.Empty
} else when (statusBeforeDownload) {
DictionaryStatus.Unknown, DictionaryStatus.Checking -> DictionaryStatus.UpToDate
else -> statusBeforeDownload
}
if (pendingSourceUrls.isNotEmpty()) {
pendingSourceUrls.forEach { urlTemplate ->
val source = dictionaryState.dictionarySources.find {
it.urlTemplate == urlTemplate
}
if (!hasDictionary) {
manageDictionarySourcesUseCase.removeSource(source.id)
if (source != null) {
val hasDictionary = dictionaryState.dictionaries.any { dictionary ->
DictionarySourceFileMatcher.matches(source, dictionary)
}
if (!hasDictionary) {
manageDictionarySourcesUseCase.removeSource(source.id)
}
}
}
pendingSourceUrls.clear()
}
pendingSourceUrls.clear()
}
}
is DownloadState.Cancelled -> {
@@ -211,6 +226,9 @@ class SettingsViewModel(
importState = operationState.importState,
dictionaryStatus = operationState.dictionaryStatus,
hasCompletedStartupScan = operationState.hasCompletedStartupScan,
pipelineIdle = !isIndexing &&
!operationState.downloadState.isActive &&
!operationState.importState.isActive,
isThemeExpanded = operationState.isThemeExpanded,
isLanguageExpanded = operationState.isLanguageExpanded,
isDictionariesExpanded = operationState.isDictionariesExpanded,
@@ -242,6 +260,9 @@ class SettingsViewModel(
android.util.Log.w("SettingsViewModel", "Failed to load language preference: ${e.message}")
}
hasCompletedStartupScan.value = true
preferencesManager.setHadNoDictionaries(
localDictionaryRepository.dictionaries.value.isEmpty()
)
}
}
}
@@ -264,26 +285,35 @@ class SettingsViewModel(
private suspend fun evaluateDictionaryStatus(): DictionaryStatus {
return try {
val actualDictionaries = localDictionaryRepository.dictionaries.first()
val hasDictionaries = actualDictionaries.isNotEmpty()
var sources = preferencesManager.dictionarySources.first()
if (pendingSourceUrls.isEmpty() &&
downloadManager.downloadState.value !is DownloadState.Loading &&
downloadManager.downloadState.value !is DownloadState.Extracting
) {
val dictionaryFileNames = localDictionaryRepository.listDictionaryPayloadFileNames(
preferencesManager.dictionaryPath
)
if (dictionaryFileNames != null) {
val installedSources = DictionarySourceFileMatcher.installedSourcesForFileNames(
sources,
dictionaryFileNames,
)
val installedSourceIds = installedSources
.mapTo(mutableSetOf(), DictionarySource::id)
val staleSourceIds = sources
.filterNot { it.id in installedSourceIds }
.map(DictionarySource::id)
if (staleSourceIds.isNotEmpty()) {
preferencesManager.removeDictionarySources(staleSourceIds)
sources = installedSources
}
}
}
if (!hasDictionaries) {
if (actualDictionaries.isEmpty()) {
return DictionaryStatus.Empty
}
var sources = preferencesManager.dictionarySources.first()
if (downloadManager.downloadState.value !is DownloadState.Loading &&
downloadManager.downloadState.value !is DownloadState.Extracting
) {
val installedSources = installedSources(sources, actualDictionaries)
val installedSourceIds = installedSources.mapTo(mutableSetOf(), DictionarySource::id)
val staleSourceIds = sources
.filterNot { it.id in installedSourceIds }
.map(DictionarySource::id)
if (staleSourceIds.isNotEmpty()) {
preferencesManager.removeDictionarySources(staleSourceIds)
sources = installedSources
}
}
val enabledSources = installedEnabledSources(sources, actualDictionaries)
if (enabledSources.isEmpty()) {
@@ -331,50 +361,69 @@ class SettingsViewModel(
}
private fun toggleDictionaryActive(dictionaryPath: String) {
localDictionaryRepository.toggleDictionaryActive(dictionaryPath)
viewModelScope.launch {
localDictionaryRepository.toggleDictionaryActive(dictionaryPath)
}
}
private fun deleteDictionary(dictionary: Dictionary) {
viewModelScope.launch {
localDictionaryRepository.deleteDictionary(dictionary)
manageDictionarySourcesUseCase.removeSourceForDictionary(dictionary)
val remainingDictionaries = localDictionaryRepository.dictionaries.first()
if (remainingDictionaries.isEmpty()) {
dictionaryStatus.value = DictionaryStatus.Empty
when (localDictionaryRepository.deleteDictionary(dictionary)) {
is OperationResult.Success -> {
val remainingDictionaryFileNames =
localDictionaryRepository.listDictionaryPayloadFileNames(
preferencesManager.dictionaryPath
)
manageDictionarySourcesUseCase.removeSourceForDictionary(
dictionary = dictionary,
remainingDictionaryFileNames = remainingDictionaryFileNames,
)
dictionaryStatus.value = withContext(Dispatchers.IO) {
evaluateDictionaryStatus()
}
}
is OperationResult.Error -> {
effectChannel.send(
getApplication<Application>().getString(R.string.error_delete_dictionary)
)
}
}
}
}
private fun addDictionarySources(urlTemplates: List<String>) {
viewModelScope.launch {
try {
val validUrls = mutableListOf<String>()
val validUrls = mutableListOf<String>()
urlTemplates.forEach { urlTemplate ->
val trimmed = urlTemplate.trim()
if (trimmed.isEmpty() || trimmed.length > 2048) {
return@forEach
}
urlTemplates.forEach { urlTemplate ->
val trimmed = urlTemplate.trim()
if (trimmed.isEmpty() || trimmed.length > 2048) {
return@forEach
}
val normalizedUrl = DictionarySource.normalizeTemplate(trimmed)
if (!pendingSourceUrls.add(normalizedUrl)) {
return@forEach
}
try {
when (manageDictionarySourcesUseCase.addSource(trimmed)) {
is ManageDictionarySourcesUseCase.AddSourceResult.Success -> {
pendingSourceUrls.add(DictionarySource.normalizeTemplate(trimmed))
validUrls.add(trimmed)
}
is ManageDictionarySourcesUseCase.AddSourceResult.ValidationFailed -> {
pendingSourceUrls.remove(normalizedUrl)
// Validation failed, skip this source
}
}
} catch (e: Exception) {
pendingSourceUrls.remove(normalizedUrl)
android.util.Log.e("SettingsViewModel", "Error adding dictionary source", e)
}
}
if (validUrls.isNotEmpty()) {
startDownloadForSources(validUrls)
}
} catch (e: Exception) {
android.util.Log.e("SettingsViewModel", "Error adding dictionary sources", e)
if (validUrls.isNotEmpty()) {
startDownloadForSources(validUrls)
}
}
}
@@ -388,6 +437,7 @@ class SettingsViewModel(
return@launch
}
statusBeforeDownload = dictionaryStatus.value
dictionaryStatus.value = DictionaryStatus.Checking
isDownloadInProgress = true
@@ -411,6 +461,7 @@ class SettingsViewModel(
return@launch
}
statusBeforeDownload = dictionaryStatus.value
dictionaryStatus.value = DictionaryStatus.Checking
val installedSources = installedEnabledSources(
@@ -446,14 +497,8 @@ class SettingsViewModel(
private fun installedSources(
sources: List<DictionarySource>,
dictionaries: List<Dictionary>
): List<DictionarySource> {
val installedFileNames = dictionaries.map { File(it.path).name }
return sources.filter { source ->
installedFileNames.any { fileName ->
DictionarySource.matchesDictionaryFile(source.urlTemplate, fileName)
}
}
}
): List<DictionarySource> =
DictionarySourceFileMatcher.installedSources(sources, dictionaries)
private fun cancelDownload() {
// Cancel the ViewModel-side scan job so no further status recomputes
@@ -499,17 +544,17 @@ class SettingsViewModel(
is OperationResult.Success -> {
val dictionaries = localDictionaryRepository.dictionaries.first()
if (dictionaries.isEmpty()) {
dictionaryStatus.value = DictionaryStatus.Empty
dictionaryStatus.value = withContext(Dispatchers.IO) {
evaluateDictionaryStatus()
}
isDownloadInProgress = false
} else if (isDownloadInProgress) {
dictionaryStatus.value = DictionaryStatus.UpToDate
isDownloadInProgress = false
} else {
if (isDownloadInProgress) {
dictionaryStatus.value = DictionaryStatus.UpToDate
isDownloadInProgress = false
} else {
dictionaryStatus.value = DictionaryStatus.Checking
val status = withContext(Dispatchers.IO) { evaluateDictionaryStatus() }
dictionaryStatus.value = status
}
dictionaryStatus.value = DictionaryStatus.Checking
val status = withContext(Dispatchers.IO) { evaluateDictionaryStatus() }
dictionaryStatus.value = status
}
if (result.data > 0) {
@@ -28,8 +28,10 @@ import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.runtime.setValue
import androidx.compose.runtime.snapshotFlow
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clipToBounds
import androidx.compose.ui.draw.scale
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.graphicsLayer
@@ -81,7 +83,8 @@ fun DictionaryListItem(
dictionary: Dictionary,
onToggle: () -> Unit,
onDelete: () -> Unit,
modifier: Modifier = Modifier
modifier: Modifier = Modifier,
isDeleteBlocked: () -> Boolean = { false }
) {
val haptic = LocalHapticFeedback.current
val density = LocalDensity.current
@@ -93,7 +96,10 @@ fun DictionaryListItem(
val offsetAnim = remember { Animatable(0f) }
var rawOffset by remember { mutableFloatStateOf(0f) }
val isDeleteRevealed by remember { derivedStateOf { rawOffset <= -swipeThresholdPx } }
val isDeleteBlockedCurrent by rememberUpdatedState(isDeleteBlocked)
val isDeleteRevealed by remember {
derivedStateOf { !isDeleteBlockedCurrent() && rawOffset <= -swipeThresholdPx }
}
LaunchedEffect(isDeleteRevealed) {
if (isDeleteRevealed) {
@@ -101,6 +107,16 @@ fun DictionaryListItem(
}
}
LaunchedEffect(Unit) {
snapshotFlow { isDeleteBlockedCurrent() }
.collect { blocked ->
if (blocked) {
offsetAnim.snapTo(0f)
rawOffset = 0f
}
}
}
val currentOnDelete by rememberUpdatedState(onDelete)
val currentOnToggle by rememberUpdatedState(onToggle)
@@ -121,6 +137,8 @@ fun DictionaryListItem(
Box(
modifier = modifier
.fillMaxWidth()
.height(DictionaryItemHeight)
.clipToBounds()
.testTag("dictionary_item")
) {
Row(
@@ -132,12 +150,15 @@ fun DictionaryListItem(
.pointerInput(maxSwipePx, swipeThresholdPx) {
detectHorizontalDragGestures(
onDragStart = {
if (isDeleteBlockedCurrent()) return@detectHorizontalDragGestures
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
},
onDragEnd = {
if (isDeleteBlockedCurrent()) return@detectHorizontalDragGestures
val target = if (rawOffset <= -swipeThresholdPx) -maxSwipePx else 0f
rawOffset = target
scope.launch {
if (isDeleteBlockedCurrent()) return@launch
offsetAnim.animateTo(
target,
animationSpec = spring(stiffness = Spring.StiffnessMedium)
@@ -145,9 +166,11 @@ fun DictionaryListItem(
}
},
onDragCancel = {
if (isDeleteBlockedCurrent()) return@detectHorizontalDragGestures
val target = if (rawOffset <= -swipeThresholdPx) -maxSwipePx else 0f
rawOffset = target
scope.launch {
if (isDeleteBlockedCurrent()) return@launch
offsetAnim.animateTo(
target,
animationSpec = spring(stiffness = Spring.StiffnessMedium)
@@ -155,9 +178,13 @@ fun DictionaryListItem(
}
},
onHorizontalDrag = { _, dragAmount ->
if (isDeleteBlockedCurrent()) return@detectHorizontalDragGestures
val newOffset = (rawOffset + dragAmount).coerceIn(-maxSwipePx, 0f)
rawOffset = newOffset
scope.launch { offsetAnim.snapTo(newOffset) }
scope.launch {
if (isDeleteBlockedCurrent()) return@launch
offsetAnim.snapTo(newOffset)
}
}
)
},
@@ -222,25 +249,11 @@ fun DictionaryListItem(
}
if (isDeleteRevealed) {
IconButton(
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
currentOnDelete()
scope.launch { offsetAnim.snapTo(0f) }
rawOffset = 0f
},
Box(
modifier = Modifier
.padding(end = ItemHorizontalPadding)
.graphicsLayer { translationX = -offsetAnim.value }
.size(IconButtonSize)
) {
Icon(
painter = painterResource(R.drawable.ic_delete),
contentDescription = stringResource(R.string.dictionary_delete),
tint = DeleteIconColor,
modifier = Modifier.size(IconSize)
)
}
)
} else {
Switch(
checked = dictionary.isActive,
@@ -257,5 +270,29 @@ fun DictionaryListItem(
)
}
}
if (isDeleteRevealed) {
IconButton(
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
currentOnDelete()
scope.launch {
offsetAnim.snapTo(0f)
rawOffset = 0f
}
},
modifier = Modifier
.align(Alignment.CenterEnd)
.padding(end = ItemHorizontalPadding)
.size(IconButtonSize)
) {
Icon(
painter = painterResource(R.drawable.ic_delete),
contentDescription = stringResource(R.string.dictionary_delete),
tint = DeleteIconColor,
modifier = Modifier.size(IconSize)
)
}
}
}
}
@@ -11,8 +11,10 @@ import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.derivedStateOf
import androidx.compose.runtime.getValue
import androidx.compose.runtime.key
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.hapticfeedback.HapticFeedbackType
@@ -21,6 +23,7 @@ import androidx.compose.ui.platform.LocalFocusManager
import androidx.compose.ui.platform.LocalHapticFeedback
import androidx.compose.ui.res.pluralStringResource
import androidx.compose.ui.res.stringResource
import androidx.compose.ui.semantics.invisibleToUser
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.unit.dp
import androidx.lifecycle.compose.collectAsStateWithLifecycle
@@ -75,6 +78,13 @@ fun DictionaryManagement(
derivedStateOf { importState is ImportState.Importing || importState is ImportState.Extracting }
}
val hasEnabledSources by remember(sources) { derivedStateOf { sources.any { it.isEnabled } } }
val isBlockingMutations = isInProgress
// Invariant: the mutation-block flag is read only at state-read/gesture time (derivedStateOf,
// snapshotFlow, pointerInput callbacks), never at row composition time. A composition-time read
// (e.g. Modifier.alpha(if (isDeleteBlocked()) ...)) would invalidate every row on each flip.
val isMutationsBlockedState = rememberUpdatedState(isBlockingMutations)
val isDeleteBlocked = remember { { isMutationsBlockedState.value } }
SectionCard(
modifier = modifier
@@ -125,13 +135,13 @@ fun DictionaryManagement(
// Disable accessibility on background content during import to improve performance.
// The progress dialog remains accessible for cancellation.
val backgroundModifier = if (isInProgress) {
Modifier.semantics(mergeDescendants = true) { }
val backgroundModifier = if (isBlockingMutations) {
Modifier.semantics { invisibleToUser() }
} else {
Modifier
}
if (dictionaries.isEmpty() && !isInProgress) {
if (dictionaries.isEmpty() && !isBlockingMutations) {
Column(
modifier = backgroundModifier
.fillMaxWidth()
@@ -153,7 +163,7 @@ fun DictionaryManagement(
)
}
} else {
if (dictionaryStatus is DictionaryStatus.NeedsUpdate && !isInProgress && hasEnabledSources) {
if (dictionaryStatus is DictionaryStatus.NeedsUpdate && !isBlockingMutations && hasEnabledSources) {
DictionaryUpdateCard(
onStartDownload = onStartDownload,
modifier = backgroundModifier
@@ -166,10 +176,11 @@ fun DictionaryManagement(
onDeleteDictionary = onDeleteDictionary,
modifier = Modifier
.fillMaxWidth()
.then(backgroundModifier)
.then(backgroundModifier),
isDeleteBlocked = isDeleteBlocked
)
if (!isInProgress) {
if (!isBlockingMutations) {
DictionaryActionButtons(
onAddSource = {
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
@@ -246,18 +257,22 @@ private fun DictionaryListSection(
dictionaries: List<Dictionary>,
onToggleDictionary: (String) -> Unit,
onDeleteDictionary: (Dictionary) -> Unit,
modifier: Modifier = Modifier
modifier: Modifier = Modifier,
isDeleteBlocked: () -> Boolean = { false }
) {
Column(
modifier = modifier,
verticalArrangement = Arrangement.spacedBy(4.dp)
) {
dictionaries.forEach { dictionary ->
DictionaryListItem(
dictionary = dictionary,
onToggle = { onToggleDictionary(dictionary.path) },
onDelete = { onDeleteDictionary(dictionary) }
)
key(dictionary.path) {
DictionaryListItem(
dictionary = dictionary,
onToggle = { onToggleDictionary(dictionary.path) },
onDelete = { onDeleteDictionary(dictionary) },
isDeleteBlocked = isDeleteBlocked
)
}
}
}
}
@@ -41,6 +41,11 @@ fun DictionaryProgressSection(
var displayedPercent by remember { mutableIntStateOf(percent) }
var displayedTestTag by remember { mutableStateOf(testTag) }
LaunchedEffect(visible) {
if (visible) {
monotonicTargetProgress = targetProgress
}
}
LaunchedEffect(visible, targetProgress) {
if (visible) {
monotonicTargetProgress = maxOf(monotonicTargetProgress, targetProgress)
+4 -1
View File
@@ -7,6 +7,7 @@
<string name="button_cancel_download">Отмена</string>
<string name="action_ok">ОК</string>
<string name="download_error">Ошибка загрузки словарей</string>
<string name="download_network_failed">Нет связи с сервером словарей. Проверьте сеть и попробуйте снова</string>
<string name="language">Язык</string>
<string name="language_english">Английский</string>
<string name="language_russian">Русский</string>
@@ -67,8 +68,9 @@
<string name="dictionary_management_title">Словари</string>
<string name="import_dictionary_button">Выбрать файлы</string>
<string name="import_error">Не удалось импортировать словарь: %1$s</string>
<string name="import_file_exists">Файл уже существует: %1$s</string>
<string name="import_file_exists">Словарь уже добавлен: %1$s</string>
<string name="import_invalid_file_name">Недопустимое имя файла: %1$s</string>
<string name="import_nothing_imported">В выбранном нет поддерживаемых файлов словарей</string>
<string name="dictionary_source_url_hint">URL</string>
<string name="dictionary_source_add_button">Добавить источник</string>
<string name="dictionary_source_duplicate">Этот URL уже существует</string>
@@ -95,6 +97,7 @@
</plurals>
<string name="dictionary_not_indexed">• Не индексирован</string>
<string name="dictionary_delete">Удалить словарь</string>
<string name="error_delete_dictionary">Не удалось удалить словарь</string>
<string name="dictionaries_tap_to_update">Нажмите для обновления</string>
<string name="notification_import_title">Импорт словарей</string>
<string name="notification_import_success_title">Импорт завершён</string>
+4 -1
View File
@@ -7,6 +7,7 @@
<string name="button_cancel_download">Cancel</string>
<string name="action_ok">OK</string>
<string name="download_error">Dictionary download failed</string>
<string name="download_network_failed">No connection to the dictionary server. Check your network and try again</string>
<string name="language">Language</string>
<string name="language_english">English</string>
<string name="language_russian">Russian</string>
@@ -65,8 +66,9 @@
<string name="dictionary_management_title">Dictionaries</string>
<string name="import_dictionary_button">Select files</string>
<string name="import_error">Failed to import dictionary: %1$s</string>
<string name="import_file_exists">File already exists: %1$s</string>
<string name="import_file_exists">Dictionary already added: %1$s</string>
<string name="import_invalid_file_name">Invalid file name: %1$s</string>
<string name="import_nothing_imported">No supported dictionary files were found in the selection</string>
<string name="dictionary_source_url_hint">URL</string>
<string name="dictionary_source_add_button">Add source</string>
<string name="dictionary_source_duplicate">This URL already exists</string>
@@ -91,6 +93,7 @@
</plurals>
<string name="dictionary_not_indexed">• Not indexed</string>
<string name="dictionary_delete">Delete dictionary</string>
<string name="error_delete_dictionary">Failed to delete dictionary</string>
<string name="dictionaries_tap_to_update">Tap to update all dictionaries</string>
<string name="notification_import_title">Importing dictionaries</string>
<string name="notification_import_success_title">Import completed</string>
@@ -0,0 +1,4 @@
- Fixed a crash and lost dictionary state around search results
- Added diacritic-insensitive search, so accented and plain spellings match each other
- Fixed indexing failures on dictionaries with dense non-Latin text
- Search results now keep loading as you scroll instead of stopping early
@@ -0,0 +1 @@
- Fixed the dictionary update card taking too long to reappear after cancelling a download
@@ -0,0 +1,2 @@
- Dictionaries can no longer be deleted while an update is running, which used to lose their update source
- Fixed the dictionary icon sliding outside the row while swiping to delete
@@ -0,0 +1 @@
- Fixed matching source URLs to installed dictionary files, so deleted dictionaries can be downloaded again from the same URL
@@ -0,0 +1,3 @@
- The dictionary update card now returns after a failed download
- Fixed headwords and articles with escaped characters being displayed incorrectly
- Fixed slow rendering of articles containing many bracket characters
@@ -9,6 +9,6 @@ Features:
- Automatic indexing with binary index files for instant search
- Multi-charset support — UTF-8, UTF-16 LE/BE auto-detection
- Adaptive three-pane layout for tablets and large screens
- Light, dark, and system-follow themes
- Material 3 UI with dynamic color, plus light, dark, and system-follow themes
- English and Russian localization
- No tracking, no ads, fully open source
Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

@@ -0,0 +1,4 @@
- Исправлен сбой и потеря состояния словаря в результатах поиска
- Добавлен поиск без учёта диакритических знаков — совпадают варианты с акцентами и без
- Исправлена ошибка индексации словарей с плотным нелатинским текстом
- Результаты поиска теперь полностью подгружаются при прокрутке, а не обрываются на первых совпадениях
@@ -0,0 +1 @@
- Исправлена лишняя задержка перед повторным появлением карточки обновления словаря после отмены загрузки
@@ -0,0 +1,2 @@
- Словари больше нельзя удалить во время обновления — раньше при этом терялся источник обновления
- Исправлен выход иконки словаря за границы строки при свайпе
@@ -0,0 +1 @@
- Исправлено сопоставление URL-источников с установленными файлами — удалённый словарь теперь можно повторно скачать по тому же URL
@@ -0,0 +1,3 @@
- Карточка обновления словарей теперь появляется снова после неудачной загрузки
- Исправлено отображение заголовков и статей с экранированными символами
- Исправлено медленное отображение статей с большим количеством скобок
@@ -9,6 +9,6 @@ ReSearch — быстрое приложение для чтения слова
- Автоматическая индексация с бинарными индексными файлами для мгновенного поиска
- Поддержка нескольких кодировок — автоопределение UTF-8, UTF-16 LE/BE
- Адаптивный трёхпанельный интерфейс для планшетов и больших экранов
- Светлая, тёмная и системная темы
- Material 3 с динамическими цветами, светлая, тёмная и системная темы
- Локализация на английский и русский языки
- Без трекеров, без рекламы, полностью открытый исходный код
Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

+6 -6
View File
@@ -1,15 +1,15 @@
[versions]
aboutlibraries = "15.0.3"
aboutlibraries = "15.2.0"
activity_compose = "1.13.0"
agp = "9.2.1"
compose_bom = "2026.06.01"
agp = "9.3.1"
compose_bom = "2026.08.00"
core = "1.19.0"
datastore_preferences = "1.2.1"
documentfile = "1.1.0"
kotlin = "2.4.0"
kotlin = "2.4.10"
lifecycle_runtime_ktx = "2.11.0"
okhttp = "5.4.0"
paging = "3.5.0"
okhttp = "5.5.0"
paging = "3.5.1"
profileinstaller = "1.4.1"
kotlinx_coroutines = "1.11.0"
kotlinx_serialization = "1.11.0"
+1 -1
View File
@@ -1,6 +1,6 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.0-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.1-bin.zip
networkTimeout=10000
retries=0
retryBackOffMs=500