http://raufibishov.com/tags/azerbaijani/Recent content in Azerbaijani on Rauf IbishovHugo -- gohugo.ioen© 2026 Rauf IbishovThu, 01 Jan 2026 00:00:00 +0000http://raufibishov.com/projects/az-neobert/Thu, 01 Jan 2026 00:00:00 +0000http://raufibishov.com/projects/az-neobert/<p><em>Status: <strong>In training</strong> · Phase 1 active on 8× NVIDIA H200 · Component 2 of the AzBERT pipeline</em></p> <h2 class="relative group">TL;DR <div id="tldr" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#tldr" aria-label="Anchor">#</a> </span> </h2> <p>I am training AzNEOBERT — an Azerbaijani encoder language model from scratch, based on the NeoBERT architecture. The model runs on 8× NVIDIA H200 GPUs via SLURM, trained on ~12.2B tokens of Azerbaijani text across 11 corpus collections. Infrastructure stack: DeepSpeed ZeRO-2, Flash Attention 3, and <code>torch.compile</code>, reaching 1.24M tokens/sec throughput. Loss dropped from 11.1 (random init) to 2.35 within the first 4,100 steps.</p>http://raufibishov.com/projects/az-tokenizer/Mon, 01 Dec 2025 00:00:00 +0000http://raufibishov.com/projects/az-tokenizer/<style> .modal { display: none; position: fixed; z-index: 1000; left: 0; top: 0; width: 100%; height: 100%; overflow: auto; background-color: #ffffff; } .modal.light-mode { background-color: #ffffff; } @media (prefers-color-scheme: dark) { .modal.dark-mode { background-color: #1a1a1a; } } .modal-content { margin: auto; padding: 40px 20px; max-width: 90vw; max-height: 90vh; overflow: auto; display: flex; align-items: center; justify-content: center; } .modal-content img { max-width: 100%; max-height: 85vh; height: auto; } .modal-close { position: absolute; top: 20px; right: 30px; font-size: 28px; font-weight: bold; cursor: pointer; color: #666; } @media (prefers-color-scheme: dark) { .modal-close { color: #999; } } .modal-close:hover { color: #000; } @media (prefers-color-scheme: dark) { .modal-close:hover { color: #fff; } } img.clickable { cursor: zoom-in; } </style> <script> function setupModals() { const modals = document.querySelectorAll('.modal'); // Close all modals function closeAllModals() { modals.forEach(modal => { modal.style.display = 'none'; }); document.body.style.overflow = 'auto'; } // Set up each image-modal pair document.getElementById('pipeline-img').addEventListener('click', (e) => { e.stopPropagation(); closeAllModals(); const modal = document.getElementById('pipeline-modal'); modal.style.display = 'block'; document.body.style.overflow = 'hidden'; applyTheme(modal); }); document.getElementById('fertility-img').addEventListener('click', (e) => { e.stopPropagation(); closeAllModals(); const modal = document.getElementById('fertility-modal'); modal.style.display = 'block'; document.body.style.overflow = 'hidden'; applyTheme(modal); }); // Set up close buttons and overlay clicks modals.forEach(modal => { const closeBtn = modal.querySelector('.modal-close'); const content = modal.querySelector('.modal-content'); closeBtn.addEventListener('click', (e) => { e.stopPropagation(); modal.style.display = 'none'; document.body.style.overflow = 'auto'; }); // Close on background click, but not on image click modal.addEventListener('click', (e) => { if (e.target === modal) { modal.style.display = 'none'; document.body.style.overflow = 'auto'; } }); // Prevent closing when clicking on the image content.addEventListener('click', (e) => { e.stopPropagation(); }); }); } function applyTheme(modal) { if (window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) { modal.classList.add('dark-mode'); modal.classList.remove('light-mode'); } else { modal.classList.add('light-mode'); modal.classList.remove('dark-mode'); } } // Initialize when DOM is ready document.addEventListener('DOMContentLoaded', setupModals); </script> <p><em>Status: <strong>Complete</strong> · WordPiece-uncased (1.727 fertility) selected for AzBERT pretraining · Component 1 of the AzBERT pipeline</em></p>