We’ve had some trouble recently with posts from aggregator links like Google Amp, MSN, and Yahoo.
We’re now requiring links go to the OG source, and not a conduit.
In an example like this, it can give the wrong attribution to the MBFC bot, and can give a more or less reliable rating than the original source, but it also makes it harder to run down duplicates.
So anything not linked to the original source, but is stuck on Google Amp, MSN, Yahoo, etc. will be removed.
You must log in or register to comment.
Haha, wow you guys really need to work on your PR
There are so many good reasons to block aggregators and you picked the worst one, your bot that no one likes.
Your bot is bad and you should feel bad
I’m open to making it better, do you have suggestions?
Everytime people try to threads either get locked, ignored or the users banned.
surprisingly admins just stick fingers in ears and yell at users to just ignore the bot
Not seeing any suggestions there to improve the bot, but lots of bannable attacks on other users, mods and admins.
So I’ll say it again, as I’ve told other people complaining, I’m open to making the bot better. If you have suggestions, I’d love to hear them.
-
It has to be automated, which means accessible through an API.
-
It has to be no/low cost. Lemmy.World doesn’t have a budget for this. We met with an MBFC alternative, they wanted 6 figures. HARD no.
How much are you paying for the MBFC API? The page says it isn’t free. I’ll give you an API endpoint which will check sources against https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Perennial_sources, if you pay me half of whatever you were paying MBFC previously. That list is quite a lot better than relying on MBFC.
I already scraped the list. It’ll take around an hour for my script to finish going down the sources and assigning web sites to each one, but I can have a working API endpoint for you tomorrow morning. I can do the bot part also, if you prefer. That’s probably easier than making a new endpoint and hooking it to a bot and debugging the connection and all.
Like I said, I think the idea that readers won’t be able to determine that Breitbart is unreliable is missing a pretty big elephant in the misinformational room. If the issue that’s causing you to keep MBFC is finding a better source that’s programmatic, though, then solving that is almost trivially easy and at least seems like some kind of step forward.
MBFC API is free as they gave us access for us as a Non Profit.
We already had in mind adding these sources to our bot but we didnt had the time and knowledge how to scrape that. Personally i would like to host it on our own server so that we dont require you to use your own money just for one bot, in what programming language did you write it?
Thanks a lot!
RookiOn a different topic: It sounds like jordanlund is saying that if he tried to remove the MBFC bot from the politics sub, he might be removed as a moderator, and replaced with someone else, and the bot would come back.
https://lemmy.world/comment/12825768
Is that true? Is the admin team mandating the use of this bot, and if so, why?
No, i dont get it from where he would get that idea, because see c/politics mods wanted the bot gone and we removed it no question asked.
@jordanlund@lemmy.world if you really dont want the bot here we can remove the bot and shut the bot down ( please consult other c/world mods too )
Here you go:
https://ponder.cat/wp/wp-sources.zip
It’s in python, suitable for sticking directly into the bot if the bot is in python. There are docs. It’s a first cut. How did you envision this working? I can make a real API, if for some reason that makes things easier, but it’s not immediately obvious how it would get integrated into things.
Running it on the last 50 articles posted to /c/politics, we see:
- https://lemmy.world/post/20739836: Source is unreliable since ownership change
- https://lemmy.world/post/20736298: Source is unreliable for political topics since 2011
- https://lemmy.world/post/20724155: Reliability consensus is mixed
- https://lemmy.world/post/20723675: Source is unreliable
- https://lemmy.world/post/20722912: Source is unreliable
- https://lemmy.world/post/20722910: Reliability consensus is mixed
- https://lemmy.world/post/20716118: Reliability consensus is mixed
- https://slrpnk.net/post/14127964: Reliability consensus is mixed
It’s more complex to use this than MBFC, because there’s a lot more depth to the rankings, and sometimes human judgement is needed to assign scores. There’s a category “needinfo,” meaning it’s necessary to know what topic is being discussed or when an article was written, because of an ownership change or similar factor. I’ve applied that judgement above. That, to me, is a good thing. It means the bot is grounded in something, and not just blithely spitting out arbitrary scores without bothering to ground them in any reality.
In practice, I think it would be realistic to assign a single reliability ranking to most of the “needinfo” sources. You can manually edit the .json data to do so. Almost all of the posts are going to fit into one of Wikipedia’s categorizations or another. Newsweek is unreliable, The Guardian is reliable, and so on.
I think most of the mixed-consensus sources can be used without a second thought. Mostly, the questions about them boil down to open partisanship of the source, which for a political community is perfectly fine as long as they’re trustable factually.
If you want me to boil this down further, so that it gives a single “yes” or “no” score to each source, I can do that and probably keep almost all of the accuracy of the rankings, now that I’ve looked at it for a little while.
When you talk about “adding” this to the bot, are you proposing to still have MBFC be the main source, with this as a footnote? A lot of the criticism of the bot is on the grounds that MBFC is a very bad source for judging reliability, so I would question the idea of keeping it on as the primary source.
By “adding” i mean adding it into the field higher than MBFC ( as i personally think wikipedia is a little bit better for that ).
new:
Wikipedia: Reliability consensus is mixed…l ( whatever the scrapper scrapes ) MBFC: Right-Center - Credibility: High - Factual Reporting: Mostly Factual - United States of America
Search Wikipedia about this sourceI would like to implement your code into the bot myself so i can learn how you would do it. If you are willing to share your code, please send me a github link ( or invite me if you want it to be private between you and me ) or if its super simple just send it in the dms.
So already ignoring. This is why people stopped giving feedback
I can’t ignore suggestions nobody is making. Have a better service in mind? Feel free to present it.
We looked at AllSides, which is good for bias, but has no scoring for credibility.
“We have to keep using the ratings website made by a random dude with no background in journalism who makes it available for free because real fact checking services cost money” is perhaps not the argument I would use for why the bot is both accurate and useful.
You don’t have to have a bot at all, especially to replace something like blacklisting Breitbart URLs, but someone thought the idea sounds cool. So “don’t have the bot” has been unnecessarily eliminated as an option. Even though sometimes the best option really is to just not have a bot.
Stop pretending that “get rid of the bot” doesn’t count as a suggestion. That’s dishonest.
I don’t even care about the bot itself, but at this point I’m just getting pissed off by all the constant distracting bickering about it.
When the question is “how do we improve it?” the answer “get rid of it” is not a genuine suggestion.
The GOOD news is, we DO have a genuinely good suggestion here and the bot creator will be reaching out.
-
Really? That’s your compelling argument?
It’s to difficult for your bot, that’s universally hated in this community, to work with?
The bot is instituted by the Admins and with good reason.
But yes, this has been the problem that pops up recently. The bot sees “Oh, MSN, they’re reliable…” but the OG source is not.
The bot is instituted by the Admins
When did this happen? The admins instituted it for !politics, and the admins changed their minds about having it for !news and friends, but wanted to keep it in !politics?
with good reason
What’s the reason?
Heightened misinformation through the election season.
Misinformation like the website MBFC, which equates the level of factual accuracy of The Guardian to Breitbart?
No it’s okay, I checked the rating for MBFC on MBFC and they rated themselves very well.
Qnr that damn left-leaning, uh, Associated Press.
MBFC does NOT equate the Guardian with Breitbart:
https://mediabiasfactcheck.com/the-guardian/
Overall, we rate The Guardian as Left-Center biased based on story selection that moderately favors the left and Mixed for factual reporting due to numerous failed fact checks over the last five years.
https://mediabiasfactcheck.com/breitbart/
Overall, we rate Breitbart Questionable based on extreme right-wing bias, the publication of conspiracy theories and propaganda, as well as numerous false claims.
If you check their list of questionable sources, Breitbart is listed, the Guardian is not:
https://mediabiasfactcheck.com/new-york-times/
Really weird it is rated “Reliable” when the New York Times wrote and reported on literal fake news weapons of mass destruction in Iraq which manufactured consent for an illegal invasion and overthrow of Iraq and killing literally MILLIONS OF PEOPLE.
On what planet is the newspaper of the establishment of the New York elite, literally wall street, “Left” I don’t think they support putting all the corporate board members in prison and establishing workers co-ops and replacing the neoliberal status quo with socialism.
The New York Times is a special class of paper called a “Newspaper of Record” until or unless that changes, nobody will question their reliability.
Example of a “failed” fact check for The Guardian:
“Private renting is making millions of people ill with almost half of England’s 8.5 million renters experiencing stress or anxiety and a quarter made physically sick as a result of their housing, campaigners have said.”
OUR VERDICT
A survey found almost a quarter of private renters agree that housing worries have made them ill in the past year. This doesn’t mean the sickness was specifically caused by renting privately as opposed to any other type of housing situation.
This was an article entirely about stress and anxiety. Ignoring that stress and anxiety have physical effects on the body, the only way someone could conclude that the article was about like, toxic apartments and not stress and anxiety was if they failed to read the article at all and instead just read the headline and made up an article in their head.
Such obviously agenda driven nitpicky bullshit is why people don’t respect the bot.
Correlation is not causation. I had my first heart attack when I was renting. It wasn’t BECAUSE I was a renter. You literally cannot say someone is experiencing stress because they’re a renter, that’s a stretch.
They could be experiencing stress by their overall socio-economic status which is also a reason they are renting, not the other way around.
I had my 2nd heart attack as a home owner. Again, my status as a renter or owner has nothing to do with it.
Jordan, please look at the ‘Factual Reporting’ metric. They consider both of them to be ‘MIXED’, and as @Catoblepas@lemmy.blahaj.zone correctly points out, the sorts of few-and-far-between “fact checks” performed on The Guardian are complete nitpicks, while Breitbart is outright a disinformation outlet, peddling climate denialism, anti-vaxx, and other things that make it – based on what you said earlier – a source that isn’t credible enough to be posted to this very community.
The Guardian is much more factually accurate than “MIXED”, and Breitbart is much less factually accurate than “MIXED”, yet somehow they elevate Breitbart while dragging The Guardian’s credibility through the mud.
(To be clear, though, I still think what you guys are doing with this change is a huge improvement.)
That’s not the overall rating though, which is why Breitbart is Questionable and the Guardian is not.
In what way does having the MediaBiasFactCheck bot help with misinformation? It’s not very accurate, probably less than the average Lemmy reader’s preexisting knowledge level. People elsewhere in these comments are posting specific examples, in a coherent, respectful fashion.
Most misinformation clearly comes in the form of accounts that post a steady stream of “reliable” articles which don’t technically break the rules, and/or in bad-faith comments. You may well be doing plenty of work on that also, I’m not saying you’re not, but it doesn’t seem from the outside like a priority in the way that the bot is. What is the use case where the bot ever helped prevent some misinformation? Do you have an example when it happened?
I’m not trying to be hostile in the way that I’m asking these questions. It’s just very strange to me that there is an overwhelming consensus by the users of this community in one direction, and that the people who are moderating it are pursuing this weird non-answer way of reacting to the overwhelming consensus. What bad thing would happen if you followed the example of the !news moderators, and just said, “You know what? We like the bot, but the community hates it, so out it goes.” It doesn’t seem like that should be a complex situation or a difficult decision, and I’m struggling to see why the moderation team is so attached to this bot and their explanations are so bizarre when they’re questioned on it.
Well, for example, just today (or maybe it was yesterday? Things get blurry after a while) somebody posted a Breitbart link.
Now, most of the Lemmy audience is smart enough to know Breitbart is bullshit, and I did remove the link when I saw it, but until I removed it, it was up with the MBFC bot making it clear to anyone who did not know that it was, in fact, bullshit.
We can’t catch everything right away, so it’s good having a bot mark these things.
Wouldn’t the fact that the Breitbart article had – if I recall correctly – a 25:10 upvote-downvote ratio by the time it was removed suggest that the MBFC bot was functionally useless in counteracting a disinformation source? Presumably because most people simply read a headline about Zelensky that wasn’t negative, said “oh cool”, upvoted without reading the article or looking at the source, and continued scrolling? And I can hardly imagine any of the 10 downvoters actually checked the MBFC bot; instead they noticed that it was Breitbart and downvoted because of its notoriety.
We discussed boiling the bot down to a tag on the posts, but apparently there was some technical limitation doing that? Frankly, it’s a little over my head.
Why does “World News” on lemmy.world give an ass about the American election season? Why was this not instituted during the elections which happened in India? We have like a billion people.
Because the winner of the US election is going to have a massive impact on the conflicts in Gaza and Ukraine.
The #1 and #2 topics in World.
Man, idk why you’re getting flak for this one.
- Certainly world@lemmy.world posts had very little overlap with the Indian elections and their voters compared to the US elections.
- I’m not sure the bot was around then.
- I think wanting to combat misinformation is a good thing; my gripe has always been that I think the bot just does a terrible job at it.
i think a better bot would be one that shows the financial/managerial ties a publisher has in a tree format so people can make the decision themselves about political bias
That actually is partially listed on the MBFC site, but you have to follow the link through from the bot to see it:
https://mediabiasfactcheck.com/the-guardian/
"Funded by / Ownership
The Guardian and its sister publication, the Sunday newspaper The Observer, are owned by Guardian Media Group plc (GMG). Scott Trust Limited was created in 1936 to ensure the editorial independence of the publications and owns Guardian Media Group plc (GMG). The Guardian states that “The Scott Trust is the sole shareholder in Guardian Media Group, and its profits are reinvested in journalism and do not benefit a proprietor or shareholders.” Donations and advertising fund the Guardian.
The Guardian switched to a tabloid print format in 2018 to cut costs. According to The New York Times, The Guardian “refused to set up a paywall — the preferred strategy of many of its rivals, from The Times of London to The Wall Street Journal and The New York Times — opting instead to ask its readers for donations, even setting up a nonprofit arm to help fund its journalism.”"
I’ve got to wonder why you guys are so insistent on the bot? Personally I just ignore it but the amount of noise it generates for you as mods cannot be worth the tiny amount of value it brings to a handful of users.
It generates a lot of noise in a thread like this, but it’a largely ignored in practical use.
If it’s usually ignored isn’t that a sign to remove it because it provides no value?
It’s definitely as value, it provides clear markers for other users before mods can intervene.
It generates a lot of noise in a thread like this, but it’a largely ignored in practical use.
provides clear markers for other users before mods can intervene.
As others have said, work on PR
Oh no, we wouldn’t want to inconvenience the mbfc bot that that literally everyone hates and wants gone. That would be awful.
I like the bot, and I also hate amp links and url forwarders that can literally take you anywhere but where it’s supposed to.
How dare you have a different opinion. It’s not hard to block the bot, I don’t see why everyone gets their jimmies so rustled about it.
The bot that everyone (inc. me ) hates?
Not everyone hates it, but if it bothers you that much, you can block it.
Just the overwhelming majority.
People overstate the hatred.
Check this post:
https://lemmy.world/post/20723122
60 upvotes, 4 comments, bot post is at +1.
Doesn’t look like an overwhelming majority, just that most people ignore it entirely.
Selection bias means that a lot of people who actively dislike the bot have it blocked.
Doesn’t mean they don’t think it’s ridiculous and misleading.
19 hours later it’s at -7. You did get good feed back. You need more sources because MBFC itself is either bad at it’s job or specifically a project to whitewash libertarian and conservative sources.
https://ponder.cat/comment/588446
One of many examples
Looks like that may be a problem with the API, not the site or the bot.
Searching the site for bbc.co.uk correctly points to the BBC:
https://mediabiasfactcheck.com/?s=bbc.co.uk
I’ll PM Rookie, maybe he needs to escape the search term or something.
Aside from the extremely vocal minority who seek it out to downvote it and complain about it constantly, it does seem like people don’t care about it when they don’t need it and appreciate it when they do. Very unscientific observation but obscure sources usually seem to have more upvotes. It doesn’t need to be useful to everyone all the time to have value.
Having quick access to MBFC and Wiki links is great and useful for mods, I assume. I also like that it carves out a thread to discuss sources. Replying to the bot makes it seem much less like you’re attacking the OP, which I always hated pre-bot.
No it’s got a bias problem. They consistently rate sources they perceive as left as less factual, consider conservative anarchists to be mainstream, and rate literal campaign websites as not very biased. They also made up their own terminology that’s loaded, despite the existence of objective terms for decades.
You’re wrong. Tons of peer-reviewed research says you’re wrong. There just isn’t any that says you’re right.
Do you have an explanation for why this bias you claim is so pervasive cannot be found when anyone looks for it? Is it… paranormal bias? Is it just really shy bias that hides when it gets scared?
How can that be true and MBFC be in broad consensus across thousands of news sites with different tools from academics, journalists, and other bias monitoring organizations? Both things cannot be true. In fact, whenever someone compares MBFC to any other resource they find almost perfect correlation, not bias. I’d love for you to explain to me where that bias disappears to when under a microscope.
Is there a conspiracy between bias monitoring organizations, journalists, and academics you have evidence of? Are the prestigious journals that published them in on it too? I can’t wait to sketch out this vast global conspiracy to pull the wool over our eyes and convince us that Democracy Now is just… highly factual. Those bastards!
That’s not saying what you want it to say. It’s a top level picture taking great pains to speak in general terms. So no, it’s not a guard against MBFC having a bias where it rates conservative stuff higher.
We’ve found concrete examples of bias in MBFC that would be very hard to see if you’re just smashing 11,000 data points against each other. This requires checking the actual sites by hand, basically doing their self appointed job again and checking their work. Then checking it against MBFCs other ratings for internal consistency.
No, if there were serious, pervasive bias impacting scores, it would lower the correlation and MBFC would be an outlier in the group because they would be in agreement less. If something’s happening at such a low level that it doesn’t impact correlation, it’s just an outlier. Multiple researchers conclude that the differences between monitors is too low to impact downstream analysis which is hard to square with your claim. And, each entry represents about 0.01% of their content, so what percentage of that data is being used to draw sweeping conclusions about the whole?
There is just high agreement about what constitutes high and low quality news sites. The notion that MBFC is somehow inferior to other bias monitors or extremely biased is not supported by evidence. If one of those organizations is better than the others, it isn’t much better. As this study concludes, because the level of agreement between them is so high, it doesn’t really matter which one you use. They’re all fine. Even they think so. Not only do MBFC ratings correlate nearly perfectly with Newsguard, Newsguard’s rating of MBFC is a perfect score. They’re well-respected by each other.
And, really, how could these researchers who’ve dedicated their lives to understanding this stuff have gotten it so wrong? Academia definitely isn’t a hotbed of conservatism. Using awful tools could destroy their careers but MBFC is regularly used in research. Why? How are these studies getting through peer-review? How are they getting published? There are just too many failure points required.
Because there’s a lot that goes into statistics. Notice I didn’t say they would be conservative, just that sometimes they can be wrong. And they take great pains to say this is a general thing. That means there’s a lot of room in the numbers. It’s not at all what you’re trying to say it is.
consider conservative anarchists
That sounds like an oxymoron. I mean there are anarcho-capitalists but most other anarchists don’t consider them anarchist.
You know them as their brand name, Libertarians. I’m making the point that they are not a mainstream center ideology as MBFC protrays.
Ok agreed.
The bot serves a very important purpose. It teaches users about the block function. I really tried to tolerate it, but it’s just like those pinned automod comments on reddit.
Just hate that it adds to the comment count
What if the Yahoo article is because the original is paywalled?
You can use an archive link to get around a paywall, that’s always been allowed.
But that’s literally rule 2.
dont use archive to get around paywalls…
Archive links are allowed, and, in fact, if you submit a link from the web interface, it offers to generate an archive link for you.
I specifically clarified that with the Admins when they asked us to crack down on copy pasting full articles.
Ah yes, people roasting volunteer mods about a thing they could easily ignore. We’re encroaching more and more on Reddit’s turf every day.
We’re not roasting the volunteer mods because we can’t ignore the bot. We’re roasting the volunteer mods because the experience of having someone in a position of power over your environment, and having them show callous indifference to how everyone in the community sees it, and what we want them to be doing with their position of power, leads people to start roasting. Sometimes out of all possible proportion to how big a deal the thing being complained about actually is.
It’s part of the healthy interplay of human society that keeps the social contract well-maintained. Take it as a sign of love, that we value this community and want it to function well.
Suuure. I’ve played this game enough to know that it doesn’t matter what decision you make, someone is going to be loudly unhappy. This is always the case; it’s not a game you can win by appealing to the true will of the shitposters, because that doesn’t exist.
Love has nothing to do with it. Some people enjoy conflict.
Downvote away, my sparkly sluts.
Why does everyone have such an issue with something that is so easily ignored? I honestly don’t understand all the outrage over this.
Because it is a bot that
- Gives a biased opinion
- Pretends to be objective
- Was not asked for, and does not need to exist
- Forced onto the users despite many objections
One could just block it if they wished, but many users feel that their only way to give real feedback is by downvoting it. The Lemmy World admins have clearly shown they will not remove it no matter how the userbase feels.
If it simply gave the admins/mods feedback about sites, there wouldn’t be very much pushback. But since it’s in every story, giving an opinion of the news organization, it is attempting to influence the conversation.
You and me both! They did have a point when the bot had a donation nag on it, the bot creator heard that complaint and removed it.
With all the negativity in here, I just wanted to say thanks for all that you do @jordanlund@lemmy.world
