reading Kiel corpus files

HOW_TO_BUILD_ONE.md

@@ -39,14 +39,12 @@ Make sure you have installed at least `make`, `gcc`, `g++` and `pkg-config` to m
 To also install a `clangarm64` toolchain (for Praat’s ARM64 edition),
 run `clangarm64.exe` to get a `clangarm64` shell. In that shell, run `pacman -Syu` to update and
 `pacman -S mingw-w64-clang-aarch64-clang` to install the build tools package.
-In the same way you can create a `clang64` toolchain and a `clang32` toolchain
-(`pacman -S mingw-w64-clang-x86_64-clang`),
-which is a good alternative to `mingw64`
-(in 2025, MSYS2 ended support for Intel32, so you can only use `mingw32` for that).
+In the same way you can create a `clang64` toolchain (`pacman -S mingw-w64-clang-x86_64-clang`).
+For Intel32, so you can only use the `mingw32` toolchain.
 
 Move the Praat sources folders somewhere in your `/home/yourname` tree,
-perhaps even in three places, e.g. as `/home/yourname/praats-arm64`,
-`/home/yourname/praats-x64` and `/home/yourname/praats-intel32`;
+perhaps even in four places, e.g. as `/home/yourname/praats-clangarm64`,
+`/home/yourname/praats-clang64v3`, `/home/yourname/praats-clang64v1` and `/home/yourname/praats-mingw32`;
 the folders `fon` and `sys` should be visible within each of these folders.
 
 If you now want to build Praat’s ARM64 edition, start the shell `clangarm64` and type
@@ -80,7 +78,7 @@ even if your computer is an ARM64 computer). You can also just set
 
     export PRAAT_ARCH=native
 
-in you `.bashrc` file or so. See the `Makefile` for details.
+in your `.bashrc` file or so. See the `Makefile` for details.
 
 **Code-signing.** From version 6.4.25 on, we have signed the three Praat executables
 with an “open-source code-signing certificate” (by Certum)
@@ -95,27 +93,26 @@ on an Intel64/AMD64 Windows 10 or 11 machine.
 
 Thorough **testing**, including of the GUI, should ultimately be done on a Windows computer,
 though `runAllTests` will successfully run on a virtual machine on an Intel64 Mac with Parallels Desktop
-(for 64-bit Windows 7, Windows 8.1, Windows 10 and Windows 11),
-or on an ARM64 Mac with Parallels Desktop (for Windows 11 only).
+(for 64-bit Windows 10 and Windows 11) or on an ARM64 Mac with Parallels Desktop (for Windows 11 only).
 Here are a couple of issues we observed, though:
 
 - For the Intel32 edition, `runAllTests.praat` doesn’t succeed on (macOS) ARM64 hardware,
   because of floating-point imprecisions; to test the Intel32 edition, use Intel64 hardware instead.
-- In December 2025, two-finger horizontal scrolling in the Sound window worked correctly in Parallels Desktop
+- Until January 2026, two-finger horizontal scrolling in the Sound window worked correctly in Parallels Desktop
   but not on a Windows computer; to reliably test the GUI, use a Windows computer instead.
 
 ## 2. Compiling for Macintosh
 
 To **build** Praat on the Mac, extract the *praatXXXX_xcodeproj.zip* file
 from [Praat’s latest release](https://github.com/praat/praat.github.io/releases)
 into the folder that contains `sys`, `fon`, `dwtools` and so on (e.g. `~/Dropbox/Praats/src`).
-Then open the project `praat.xcodeproj` in Xcode 26.1.1 (or later),
+Then open the project `praat.xcodeproj` in Xcode 26.6 (or later),
 and edit the Intermediate and Product build paths to something that suits you
 (Xcode -> Settings... -> Locations -> Derived Data -> Advanced... -> Custom -> Absolute,
 then type something after Products, e.g. `~/Dropbox/Praats/bin/macos`,
 as well as something after Intermediates, e.g. `~/builds/mac_intermediates`, then click Done).
 After this preliminary work, choose Build or Run for the target `praat_mac`.
-You can compile with the 14.2 SDK, which will work as far back as macOS 10.11 El Capitan,
+You can compile with the 26.5 SDK, which will work as far back as macOS 10.15 Catalina,
 which is our deployment target, and will look good even on macOS 26 Tahoe.
 
 If you get an error message like “Code Signing Identity xxx does not match any valid, non-expired,
@@ -147,10 +144,8 @@ and or go (or log in) to App Store Connect, then Business (or Agreements, Tax, a
 -> Paid Apps Agreement -> View and Agree to Terms (even if you have no paid apps).
 
 **Testing** on multiple Intel64/AMD64 platform versions can be done on older Intel64 Macs,
-using virtual machines with Parallels Desktop. For instance, a 2013 Macbook Pro can handle
-OS X 10.11 El Capitan, 10.12 Sierra, 10.13 High Sierra, macOS 10.14 Mojave, 10.15 Catalina,
-and macOS 11 Big Sur, while a 2018 Macbook Pro can handle macOS 10.14 Mojave, 10.15 Catalina,
-macOS 11 Big Sur, macOS 12 Monterey, and macOS 14 Sonoma natively.
+using virtual machines with Parallels Desktop. For instance, a 2018 Macbook Pro can handle
+macOS 10.15 Catalina, macOS 11 Big Sur, macOS 12 Monterey, and macOS 14 Sonoma natively.
 Testing on multiple ARM64 platform versions can be done on an older ARM64 Mac,
 using virtual machines with Parallels Desktop. For instance, a 2020 Mac Mini could handle
 macOS 11 Big Sur, and macOS 12 Monterey, and macOS 13 Ventura (and macOS 14 Sonoma natively),

dwtools/praat_David_init.cpp

@@ -9014,7 +9014,6 @@ void praat_David_generics_new_init () {
 void praat_David_init ();
 void praat_David_init () {
 
-	Data_recognizeFileType (TextGrid_TIMITLabelFileRecognizer);
 	Data_recognizeFileType (cmuAudioFileRecognizer);
 	Data_recognizeFileType (oggFileRecognizer);
 

fon/Corpus.cpp

@@ -204,7 +204,7 @@ autoCorpus Corpus_extractComponents (Corpus me,
 	return thee;
 }
 
-autoCorpus Corpus_extractEducationRegions (Corpus me,
+autoCorpus Corpus_extractSpeakerEducationRegions (Corpus me,
 	bool sHolland, bool nHolland, bool wUtrecht, bool zeeland, bool eUtrecht,
 	bool guelders, bool veluwe, bool wFriesland, bool polders, bool achterhoek,
 	bool overijssel, bool drenthe, bool groningen, bool frisia, bool nBrabant,
@@ -226,28 +226,34 @@ autoCorpus Corpus_extractEducationRegions (Corpus me,
 			IntervalTier tier = (IntervalTier) textgrid -> tiers->at [itier];   // TODO: check cast
 			conststring32 tierName = tier -> name.get();
 			if (Melder_stringMatchesCriterion (tierName, kMelder_string::STARTS_WITH, U"ort/", true)) {
-				const conststring32 speakerName = & tier -> name [4];
-				integer speakerRow = Table_searchColumn (thy speakers.get(), 5, speakerName);
+				static MelderString speakerName;
+				MelderString_copy (& speakerName, & tier -> name [4]);
+				integer speakerRow = Table_searchColumn (thy speakers.get(), 5, speakerName.string);
 				if (speakerRow == 0) {
-					if (! str32equ (speakerName, U"BACKGROUND") &&
-						! str32equ (speakerName, U"COMMENT") &&
-						! str32equ (speakerName, U"UNKNOWN") &&
-						! str32equ (speakerName, U"N00383"))
-						Melder_crash (U"<<", speakerName, U">> <<", tierName, U">>");
+					if (! str32equ (speakerName.string, U"BACKGROUND") &&
+						! str32equ (speakerName.string, U"COMMENT") &&
+						! str32equ (speakerName.string, U"UNKNOWN") &&
+						! str32equ (speakerName.string, U"N00383"))
+						Melder_crash (U"<<", speakerName.string, U">> <<", tierName, U">>");
 				} else {
 					conststring32 speakerRegionCode = Table_getStringValue_a (thy speakers.get(), speakerRow, 17);
 					integer icode = 0;
 					for (; icode < 25; icode ++)
 						if (str32equ (speakerRegionCode, Corpus_CGN_regionCodes [icode]))
 							break;
 					if (icode >= 25) {
-						if (str32equ (speakerRegionCode, U"regB"))
+						if (str32equ (speakerRegionCode, U"regB"))   // this fixes a mistake in the CGN_2.0.3 speakers table; turn into "other"
 							icode = 24;
 					}
 					if (icode >= 25)
 						Melder_crash (icode, U" <<", speakerRegionCode, U">>");
 					if (regions [icode]) {
 						foundMatch = true;
+						for (integer jtier = 1; jtier <= textgrid -> tiers->size; jtier ++) {
+							IntervalTier tierj = (IntervalTier) textgrid -> tiers->at [jtier];   // TODO: check cast
+							if (str32str (tierj -> name.get(), speakerName.string))
+								Thing_setName (tierj, Melder_cat (tierj -> name.get(), U"(", Corpus_CGN_regions_abbreviated [icode], U")"));
+						}
 						break;
 					}
 				}

fon/Corpus.h

@@ -54,6 +54,14 @@ static conststring32 Corpus_CGN_regions_short [25] {
 	U"W-Limburg", U"N-Belgium", U"S-Belgium", U"reg_other", U"reg_unknown"
 };
 
+static conststring32 Corpus_CGN_regions_abbreviated [25] {
+	U"SH", U"NH", U"WU", U"Ze", U"EU",
+	U"Ge", U"Ve", U"WF", U"Po", U"Ac",
+	U"Ov", U"Dr", U"Gr", U"Fr", U"NB",
+	U"EL", U"N", U"SB", U"EV", U"WV",
+	U"WL", U"No", U"So", U"ot", U"un"
+};
+
 static conststring32 Corpus_CGN_regions_long [25] {
 	U"South Holland (province of Zuid-Holland excl. Goeree-Overflakkee)",
 	U"North Holland (province of Noord-Holland excl. West-Friesland)",
@@ -82,7 +90,7 @@ static conststring32 Corpus_CGN_regions_long [25] {
 	U"unknown region"
 };
 
-autoCorpus Corpus_extractEducationRegions (Corpus me,
+autoCorpus Corpus_extractSpeakerEducationRegions (Corpus me,
 	bool sHolland, bool nHolland, bool wUtrecht, bool zeeland, bool eUtrecht,
 	bool guelders, bool veluwe, bool wFriesland, bool polders, bool achterhoek,
 	bool overijssel, bool drenthe, bool groningen, bool frisia, bool nBrabant,

fon/TextGrid.h

@@ -215,6 +215,10 @@ autoDaata TextGrid_TIMITLabelFileRecognizer (integer nread, const char *header,
 		A valid word is a string with contains the lowercase characters [a-z] and ['].
 */
 
+autoDaata TextGrid_KielLabelFileRecognizer (integer nread, const char *header, MelderFile file);
+autoIntervalTier IntervalTier_readFromKielLabelFile (MelderFile file);
+autoTextGrid TextGrid_readFromKielLabelFile (MelderFile file);
+
 void TextGrid_checkInvariants_e (const constTextGrid me, const bool includeWeakInvariants);
 
 void TextGrid_scaleTimes_e (mutableTextGrid me, double xminfrom, double xmaxfrom, double xminto, double xmaxto);

fon/TextGrid_Sound.cpp

@@ -1449,4 +1449,61 @@ autoTextGrid TextGrid_Sound_readFromCorpusGesprokenNederlands (conststring32 sou
 	}
 }
 
+autoTextGrid TextGrid_Sound_readFromKielCorpus (conststring32 soundFileName, autoSound *out_sound) {
+	try {
+		structMelderFile file { };
+		Melder_pathToFile (soundFileName, & file);
+		if (out_sound)
+			*out_sound = Sound_readFromSoundFile (& file);
+		OrderedOf <structTextGrid> textgrids;
+
+		/*
+			The sound file resides in a folder like "~/Dropbox/Corpora/Kiel/Vol4_DVD1/l01/chan_cje.
+			We extract the name of the sound file and of the enclosing folders.
+		*/
+
+		conststring32 soundName = MelderFile_name (& file);
+		const integer soundNamelength = Melder_length (soundName);
+		Melder_require (soundNamelength == 10,
+			U"Sound file name should be 10 characters long, but “", soundName, U"” is ", soundNamelength, U" characters long.");
+		Melder_require (Melder_stringMatchesCriterion (soundName, kMelder_string::ENDS_WITH, U".wav", true),
+			U"Sound file name should end in “.wav”, but it is “", soundName, U"”.");
+
+		autostring32 baseName = Melder_dup (soundName);
+		baseName [soundNamelength - 4] = U'\0';   // remove extension ".wav"
+
+		/*
+			Travel up to the speaker folder.
+		*/
+
+		structMelderFolder speakerFolder { };
+		MelderFile_getParentFolder (& file, & speakerFolder);
+		conststring32 speakerName = MelderFolder_name (& speakerFolder);
+
+		/*
+			Read the .s1h file.
+		*/
+		structMelderFile annotationFile { };
+		MelderFolder_getFile (& speakerFolder, Melder_cat (baseName.get(), U".s1h"), & annotationFile);
+		autoDaata annotationData = Data_readFromFile (& annotationFile);
+		Melder_require (Thing_isa (annotationData.get(), classTextGrid),
+			U"The file ", MelderFile_messageName (& annotationFile), U" contains a ", Thing_className (annotationData.get()), U" instead of a TextGrid.");
+		autoTextGrid ort = annotationData.static_cast_move <structTextGrid>();
+		for (integer itier = 1; itier <= ort -> tiers->size; itier ++)
+			TextGrid_setTierName (ort.get(), itier, Melder_dup (Melder_cat (U"ort/", ort -> tiers->at [itier] -> name.get())).get());
+		textgrids. addItem_ref (ort.get());
+
+		autoTextGrid me = TextGrids_merge (& textgrids, true);
+		for (integer itier = 1; itier <= my tiers->size; itier ++)
+			TextGrid_setTierName (me.get(), itier, replace_STR (my tiers->at [itier] -> name.get(), U"_", U"/", 0).get());
+
+		if (out_sound)
+			Thing_setName (out_sound->get(), baseName.get());
+		Thing_setName (me.get(), baseName.get());
+		return me;
+	} catch (MelderError) {
+		Melder_throw (U"Sound “", soundFileName, U"” not read with adjacent Kiel annotation files.");
+	}
+}
+
 /* End of file TextGrid_Sound.cpp */

fon/TextGrid_Sound.h

@@ -54,5 +54,6 @@ void TextGrid_Sound_diarizeInterval (TextGrid me, Sound sound, integer tierNumbe
 autoSound Sound_readWithAdjacentAnnotationFiles_buckeye (conststring32 soundFileName, autoTextGrid *out_textgrid);
 autoSound Sound_readWithAdjacentAnnotationFiles_timit   (conststring32 soundFileName, autoTextGrid *out_textgrid);
 autoTextGrid TextGrid_Sound_readFromCorpusGesprokenNederlands   (conststring32 soundFileName, autoSound *out_Sound);
+autoTextGrid TextGrid_Sound_readFromKielCorpus   (conststring32 soundFileName, autoSound *out_Sound);
 
 /* End of file TextGrid_Sound.h */

fon/TextGrid_files.cpp

@@ -1050,4 +1050,99 @@ autoTextGrid TextGrid_readFromTimitLabelFile (
 	}
 }
 
+autoDaata TextGrid_KielLabelFileRecognizer (integer nread, const char *header, MelderFile file) {
+	conststring32 fileName = MelderFile_name (file);
+	if (nread < 512 || ! Melder_endsWith_caseAware (fileName, U".s1h"))
+		return autoDaata ();
+	autoTextGrid thee = TextGrid_readFromKielLabelFile (file);
+	return thee.move();
+}
+
+autoIntervalTier IntervalTier_readFromKielLabelFile (
+	const MelderFile file
+) {
+	try {
+		autoMelderReadText text = MelderReadText_createFromFile (file);   // going to be UTF-8-compatible
+		constexpr double fixedTimitSamplingFrequency = 16000.0;   // Hz
+		constexpr double fixedSamplingPeriod = 1.0 / fixedTimitSamplingFrequency;
+		autoMelderString label;
+		autoIntervalTier me = IntervalTier_create_raw (+1e308, -1e308);
+		Melder_assert (my intervals.size == 0);
+
+		/* mutable step */ integer previousStartingSample = -2;
+		/* mutable found */ bool annotationLinesFound = false;
+		/* mutable step */ double previousStartingTime = undefined;
+		for (;;) {
+			/* mutable scan */ conststring32 line = MelderReadText_readLine (text.get());
+			if (! line)
+				break;
+			if (! annotationLinesFound) {
+				if (Melder_equ (line, U"hend"))
+					annotationLinesFound = true;
+				continue;
+			}
+
+			/*
+				Read and check starting sample.
+			*/
+			Melder_skipHorizontalSpace (& line);
+			const integer startingSample = Melder_readInteger (& line);
+			Melder_require (startingSample > 0,
+				U"The annotation should not start at a negative sample number.");
+			Melder_assert (startingSample > 0);
+			if (previousStartingSample == -2) {
+				previousStartingSample = startingSample;
+				previousStartingTime = previousStartingSample * fixedSamplingPeriod;
+				continue;
+			}
+			Melder_require (startingSample >= previousStartingSample,
+				U"The starting sample in line ", MelderReadText_getLineNumber (text.get()) - 1,
+				U" should be at least the starting sample of the previous line, but ",
+				startingSample, U" is less than ", previousStartingSample, U"."
+			);
+			{// scope
+				const char32 shouldBeHorizontalSpace = * line ++;
+				Melder_require (Melder_isHorizontalSpace (shouldBeHorizontalSpace),
+					U"There should be a space after the starting sample in line ", MelderReadText_getLineNumber (text.get()) - 1, U".");
+			}
+
+			/*
+				Read and check label text.
+			*/
+			const double startingTime = startingSample * fixedSamplingPeriod;
+			if (startingTime != previousStartingTime) {
+				//Melder_casual (previousStartingTime, U" ", startingTime, U" ", label.string);
+				char32* space = str32chr (label.string, U' ');
+				if (space)
+					*space = U'\0';
+				(void) IntervalTier_addInterval_raw (me.get(), previousStartingTime, startingTime, label.string);
+			}
+			Melder_skipHorizontalSpace (& line);
+			MelderString_copy (& label, line);
+
+			previousStartingSample = startingSample;
+			previousStartingTime = startingTime;
+		}
+		Melder_require (my intervals.size >= 1,
+			U"The file should contain at least one interval, but none were found.");
+		return me;
+	} catch (MelderError) {
+		Melder_throw (U"Interval tier not read from file ", file, U".");
+	}
+}
+
+autoTextGrid TextGrid_readFromKielLabelFile (
+	const MelderFile file
+) {
+	try {
+		autoTextGrid result = TextGrid_createWithoutTiers (+1e308, -1e308);
+		autoIntervalTier tier = IntervalTier_readFromKielLabelFile (file);
+		TextGrid_addTier_move (result.get(), tier.move());
+		return result;
+	} catch (MelderError) {
+		Melder_throw (U"TextGrid not read from file ", file, U".");
+	}
+}
+
+
 /* End of file TextGrid_files.cpp */