We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model at least as good as GPT-3 (davinci) ...and unveil how models of such a scale can be successfully pre-trained. Over the course of this effort, we face numerous unexpected technical and engineering challenges, particularly on loss spikes and divergence. In this paper, we introduce the training process of GLM-130B including its design choices, training strategies for both efficiency and stability, and engineering efforts. The resultant GLM-130B model offers significant outperformance over GPT-3 175B (davinci) on a wide range of popular English benchmarks while the performance advantage is not observed in OPT-175B and BLOOM-176B. It also consistently and significantly outperforms ERNIE TITAN 3.0 260B -- the largest Chinese language model -- across related benchmarks. Finally, we leverage a unique scaling property of GLM-130B to reach INT4 quantization without post training, with almost no performance loss, making it the first among 100B-scale models and more importantly, allowing its effective inference on 4\(\times\)RTX 3090 (24G) or 8\(\times\)RTX 2080 Ti (11G) GPUs, the most affordable GPUs required for using 100B-scale models. The GLM-130B model weights are publicly accessible and its code, training logs, related toolkit, and lessons learned are open-sourced at \url{https://github.com/THUDM/GLM-130B/}.
IntroductionHepatic inflammatory myofibroblastic tumor (HIMT) is a junctional neoplastic lesion of mesenchymal tissue origin that can sometimes become locally invasive and even metastasize or recur. ...Therefore, the diagnosis and treatment of HIMT is particularly important. However, hepatic inflammatory myofibroblastic tumor lacks a specific clinical presentation and typical imaging manifestations, thus posing a difficulty for us to diagnose and treat this disease. Case PresentationWe report here a very rare surgical case of hepatic inflammatory myofibroblastic tumor (HIMT) in a 41-year-old female who was admitted to the hospital for more than half a month for a liver-occupying lesion with fever found on physical examination.After discussion with the hepatobiliary and pancreatic surgery team, we decided to perform surgical treatment. The final postoperative pathology confirmed hepatitis myofibroblastoma. ConclusionOur review of the domestic and international literature revealed no significant progress in the diagnosis and treatment of this disease, so we report here a case of surgical treatment. One of our aims in this case report is to highlight the efficacy of surgical treatment in HIMT. HIMT is extremely rare and difficult to diagnose. Due to their intermediate biological behavior, surgical resection should be performed whenever feasible and patients should be followed-up in order to detect recurrence and metastasis as early as possible.
The ability to automatically detect and track surgical instruments in endoscopic videos can enable transformational interventions. Assessing surgical performance and efficiency, identifying skilled ...tool use and choreography, and planning operational and logistical aspects of OR resources are just a few of the applications that could benefit. Unfortunately, obtaining the annotations needed to train machine learning models to identify and localize surgical tools is a difficult task. Annotating bounding boxes frame-by-frame is tedious and time-consuming, yet large amounts of data with a wide variety of surgical tools and surgeries must be captured for robust training. Moreover, ongoing annotator training is needed to stay up to date with surgical instrument innovation. In robotic-assisted surgery, however, potentially informative data like timestamps of instrument installation and removal can be programmatically harvested. The ability to rely on tool installation data alone would significantly reduce the workload to train robust tool-tracking models. With this motivation in mind we invited the surgical data science community to participate in the challenge, SurgToolLoc 2022. The goal was to leverage tool presence data as weak labels for machine learning models trained to detect tools and localize them in video frames with bounding boxes. We present the results of this challenge along with many of the team's efforts. We conclude by discussing these results in the broader context of machine learning and surgical data science. The training data used for this challenge consisting of 24,695 video clips with tool presence labels is also being released publicly and can be accessed at https://console.cloud.google.com/storage/browser/isi-surgtoolloc-2022.
BackgroundO'Donnell-Luria-Rodan (ODLURO) syndrome is an autosomal dominant systemic disorder characterized by global developmental delay caused by mutations in the KMT2E gene. The aim of this study ...was to investigate the role of KMT2E mutations as a cause of ODLURO syndrome in a Chinese boy. MethodsWe reported the clinical course of a Chinese boy who was diagnosed with ODLURO syndrome by the whole exome sequencing. We extracted genomic DNA of the proband and parents, gene variations were screened using whole-exome sequencing, followed by validation using direct Sanger sequencing. The effect of mRNA splicing variants were analyzed through a minigene splice assay and in vitro reverse transcription PCR (RT-PCR). ResultsThe proband presented with recurrent seizures and developmental delay. Using genetic analysis, we identified that the proband carried a de novo heterozygous splicing variant (c.1248+1G>T) in the KMT2E gene. In vivo transcript analysis showed that the proband did not carry any KMT2E mRNA transcript, while a specific exon11-exon13 (440 bp) transcript was detected in the unaffected parents. The in vitro minigene splice assay conducted in HEK293 cells confirmed that the c.1248+1G>T variant resulted in exon 12 skipping, which in turn caused an alteration in KMT2E mRNA splicing. The mutant transcript created a premature stop codon at the 378 amino acid position that could have been caused nonsense-mediated mRNA decay (NMD). ConclusionWe verified the pathogenic effect of the KMT2E c.1248+1G>T splicing variant, which disturbed normal mRNA splicing and caused mRNA decay. Our findings suggest that splice variants play an important role in the molecular basis of ODLURO, and that careful molecular profiling of these patients could play an essential role in tailoring of personalized treatment options soon.
Context-aware decision support in the operating room can foster surgical safety and efficiency by leveraging real-time feedback from surgical workflow analysis. Most existing works recognize surgical ...activities at a coarse-grained level, such as phases, steps or events, leaving out fine-grained interaction details about the surgical activity; yet those are needed for more helpful AI assistance in the operating room. Recognizing surgical actions as triplets of <instrument, verb, target> combination delivers comprehensive details about the activities taking place in surgical videos. This paper presents CholecTriplet2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. The challenge granted private access to the large-scale CholecT50 dataset, which is annotated with action triplet information. In this paper, we present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods from the challenge organizers and 19 new deep learning algorithms by competing teams are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%. This study also analyzes the significance of the results obtained by the presented approaches, performs a thorough methodological comparison between them, in-depth result analysis, and proposes a novel ensemble method for enhanced recognition. Our analysis shows that surgical workflow analysis is not yet solved, and also highlights interesting directions for future research on fine-grained surgical activity recognition which is of utmost importance for the development of AI in surgery.
PDF