Introduction: Most people with psychiatric illnesses do not receive treatment for almost a decade after disorder onset. Online mental health screens reflect one mechanism designed to shorten this lag in help-seeking, yet there has been limited research on the effectiveness of screening tools in naturalistic settings. Material and methods: We examined a cohort of persons directed to a mental health screening tool via the Bing search engine (n = 126,060). We evaluated the impact of tool content on later searches for mental health selfreferences, self-diagnosis, care seeking, psychoactive medications, suicidal ideation, and suicidal intent. Website characteristics were evaluated by pairs of independent raters to ascertain screen type and content. These included the presence/absence of a suggestive diagnosis, a message on interpretability, as well as referrals to digital treatments, in-person treatments, and crisis services. Results: Using machine learning models, the results suggested that screen content predicted later searches with mental health self-references (AUC = 0.73), mental health self-diagnosis (AUC = 0.69), mental health care seeking (AUC = 0.61), psychoactive medications (AUC = 0.55), suicidal ideation (AUC = 0.58), and suicidal intent (AUC = 0.60). Cox-proportional hazards models suggested individuals utilizing tools with in-person care referral were significantly more likely to subsequently search for methods to actively end their life (HR = 1.727,p = 0.007). Discussion: Online screens may influence help-seeking behavior, suicidal ideation, and suicidal intent. Websites with referrals to in-person treatments could put persons at greater risk of active suicidal intent. Further evaluation using large-scale randomized controlled trials is needed.