Skip to content

Commit

Permalink
Merge branch 'sherlock-project:master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
@b34c0n5
b34c0n5 committed May 10, 2024
2 parents 0547417 + f794e23 commit 54597f3
Show file tree
Hide file tree
Showing 11 changed files with 456 additions and 235 deletions.
7 changes: 3 additions & 4 deletions .github/workflows/pull_request.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,12 @@ jobs:
- name: Install Dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
pip install ruff flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint With flake8
- name: Lint With Ruff
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
ruff check . --output-format=github --select=E9,F63,F7,F82
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Sherlock Site Detect Tests
Expand Down
87 changes: 37 additions & 50 deletions removed_sites.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,13 +87,6 @@
"urlMain": "https://kiwifarms.net/",
"username_claimed": "blue"
},
"Linkedin": {
"errorMsg": "could not be found",
"errorType": "message",
"url": "https://www.linkedin.com/in/{}",
"urlMain": "https://www.linkedin.com/",
"username_claimed": "alex"
},
"NPM-Package": {
"errorType": "status_code",
"url": "https://www.npmjs.com/package/{}",
Expand Down Expand Up @@ -492,13 +485,6 @@
"urlMain": "https://www.cloob.com/",
"username_claimed": "blue"
},
"1337x": {
"errorMsg": "Bad Username",
"errorType": "message",
"url": "https://1337x.to/user/{}/",
"urlMain": "https://1337x.to",
"username_claimed": "TheMorozko"
},
"TM-Ladder": {
"errorMsg": "player unknown or invalid",
"errorType": "message",
Expand Down Expand Up @@ -573,13 +559,6 @@
"urlMain": "https://open.spotify.com/",
"username_claimed": "blue"
},
"Steam": {
"errorMsg": "The specified profile could not be found",
"errorType": "message",
"url": "https://steamcommunity.com/id/{}",
"urlMain": "https://steamcommunity.com/",
"username_claimed": "blue"
},
"Raidforums": {
"errorType": "status_code",
"url": "https://raidforums.com/User-{}",
Expand Down Expand Up @@ -763,14 +742,6 @@
"urlProbe": "https://onlyfans.com/api2/v2/users/{}",
"username_claimed": "theemilylynne"
},
"Instagram": {
"errorMsg": "Nothing found!",
"errorType": "message",
"url": "https://www.instagram.com/{}",
"urlMain": "https://www.instagram.com/",
"urlProbe": "https://www.picuki.com/profile/{}",
"username_claimed": "blue"
},
"OK": {
"errorType": "status_code",
"regexCheck": "^[a-zA-Z][a-zA-Z0-9_.-]*$",
Expand Down Expand Up @@ -811,27 +782,6 @@
"urlMain": "https://quizlet.com",
"username_claimed": "blue"
},
"Youtube Channel": {
"errorCode": 404,
"errorType": "status_code",
"headers": {
"Cookie": "CONSENT=YES+cb.20210418-17-p0.it+FX+917; "
},
"url": "https://www.youtube.com/c/{}",
"urlMain": "https://www.youtube.com",
"username_claimed": "mkbhd"
},
"Youtube User": {
"errorCode": 404,
"errorType": "status_code",
"headers": {
"Cookie": "CONSENT=YES+cb.20210418-17-p0.it+FX+917; "
},
"url": "https://www.youtube.com/user/{}",
"urlMain": "https://www.youtube.com",
"username_claimed": "pewdiepie",
"username_unclaimed": "noonewouldeverusethis7"
},
"GunsAndAmmo": {
"errorType": "status_code",
"url": "https://forums.gunsandammo.com/profile/{}",
Expand Down Expand Up @@ -869,5 +819,42 @@
"url": "https://ebio.gg/{}",
"urlMain": "https:/ebio.gg",
"username_claimed": "dev"
},
"metacritic": {
"errorMsg": "User not found",
"errorType": "message",
"regexCheck": "^(?![-_].)[A-Za-z0-9-_]{3,15}$",
"url": "https://www.metacritic.com/user/{}",
"urlMain": "https://www.metacritic.com/",
"username_claimed": "blue"
},
"Oracle Communities": {
"errorType": "status_code",
"url": "https://community.oracle.com/people/{}",
"urlMain": "https://community.oracle.com",
"username_claimed": "dev"
},
"HexRPG": {
"errorMsg": "Error : User ",
"errorType": "message",
"regexCheck": "^[a-zA-Z0-9_ ]{3,20}$",
"url": "https://www.hexrpg.com/userinfo/{}",
"urlMain": "https://www.hexrpg.com/",
"username_claimed": "blue"
},
"G2G": {
"errorType": "response_url",
"errorUrl": "https://www.g2g.com/{}",
"regexCheck": "^[A-Za-z][A-Za-z0-9_]{2,11}$",
"url": "https://www.g2g.com/{}",
"urlMain": "https://www.g2g.com/",
"username_claimed": "user"
},
"BitCoinForum": {
"errorMsg": "The user whose profile you are trying to view does not exist.",
"errorType": "message",
"url": "https://bitcoinforum.com/profile/{}",
"urlMain": "https://bitcoinforum.com",
"username_claimed": "bitcoinforum.com"
}
}
139 changes: 62 additions & 77 deletions removed_sites.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -339,13 +339,6 @@ user names were available.
},
```

## LinkedIn

This was attempted to be added around 2019-08-26, but the pull request was never merged.
It turns out that LinkedIn requires that you have an account before they will let you
check for other account. So, this site will not work with the current design of
Sherlock.

## StreamMe

On 2019-04-07, I get a Timed Out message from the website. It has not
Expand Down Expand Up @@ -1178,19 +1171,6 @@ As of 2021-10-25, Cloob seems to be down and their site is not responding.
}
```

### 1337x
As of 2021-11-21, 1337x seems to be down causing false positives.
```json
"1337x": {
"errorMsg": "Bad Username",
"errorType": "message",
"url": "https://1337x.to/user/{}/",
"urlMain": "https://1337x.to",
"username_claimed": "TheMorozko",
"username_unclaimed": "noonewouldeverusethis7"
}
```

### TM-Ladder
As of 2021-11-30, TM-Ladder is returning false positives due to rate limits.

Expand Down Expand Up @@ -1331,20 +1311,6 @@ As og 2022-05-01, Countable returns false positives
},
```

## Steam

As og 2022-05-01, Steam returns false positives
```json
"Steam": {
"errorMsg": "The specified profile could not be found",
"errorType": "message",
"url": "https://steamcommunity.com/id/{}",
"urlMain": "https://steamcommunity.com/",
"username_claimed": "blue",
"username_unclaimed": "noonewouldeverusethis7"
},
```

## Raidforums

Raidforums is [now run by the FBI](https://twitter.com/janomine/status/1499453777648234501?s=21)
Expand Down Expand Up @@ -1700,20 +1666,6 @@ As of 2023.04.20, OnlyFans returns false negatives on checking usernames with th
}
```

## Instagram
As of 2023.04.21, Instagram returns false positives as picuki.com was used to query for usernames but they now user Cloudflare

```json
"Instagram": {
"errorMsg": "Nothing found!",
"errorType": "message",
"url": "https://www.instagram.com/{}",
"urlMain": "https://www.instagram.com/",
"urlProbe": "https://www.picuki.com/profile/{}",
"username_claimed": "blue"
}
```

## OK
As of 2023.04.21, Ok.ru returns false positives
```json
Expand Down Expand Up @@ -1786,35 +1738,6 @@ As of 2023.08.29 Quizlet requires us to enable JavaScript to check if a user exs
}
```


## YouTube
As of 2023.08.29, YouTube returns false positives as we need to accept their TOC but thats not possible with how Sherlock currently works


```json
"Youtube Channel": {
"errorCode": 404,
"errorType": "status_code",
"headers": {
"Cookie": "CONSENT=YES+cb.20210418-17-p0.it+FX+917; "
},
"url": "https://www.youtube.com/c/{}",
"urlMain": "https://www.youtube.com",
"username_claimed": "mkbhd"
},
"Youtube User": {
"errorCode": 404,
"errorType": "status_code",
"headers": {
"Cookie": "CONSENT=YES+cb.20210418-17-p0.it+FX+917; "
},
"url": "https://www.youtube.com/user/{}",
"urlMain": "https://www.youtube.com",
"username_claimed": "pewdiepie",
"username_unclaimed": "noonewouldeverusethis7"
}
```

## GunsAndAmmo
As of 2023.08.29, GunsAndAmmo responds with 404 from time to time
```json
Expand Down Expand Up @@ -1882,4 +1805,66 @@ As of 2023.12.21, Ebio returns false positives.
"urlMain": "https:/ebio.gg",
"username_claimed": "dev"
},
```

## HexRPG
__2024-04-07 :__ HexRPG behind authentication wall. Unable to check usernames without logging in.
```json
"HexRPG": {
"errorMsg": "Error : User ",
"errorType": "message",
"regexCheck": "^[a-zA-Z0-9_ ]{3,20}$",
"url": "https://www.hexrpg.com/userinfo/{}",
"urlMain": "https://www.hexrpg.com/",
"username_claimed": "blue"
}
```

## Oracle Communities
__2024-04-07 :__ Oracle Communities behind authentication wall. Unable to check usernames without logging in.
```json
"Oracle Communities": {
"errorType": "status_code",
"url": "https://community.oracle.com/people/{}",
"urlMain": "https://community.oracle.com",
"username_claimed": "dev"
}
```

## Metacritic
__2024-04-07 :__ Non-existent users seemingly displayed as real users with no activity. Needs adjustment.
```json
"metacritic": {
"errorMsg": "User not found",
"errorType": "message",
"regexCheck": "^(?![-_].)[A-Za-z0-9-_]{3,15}$",
"url": "https://www.metacritic.com/user/{}",
"urlMain": "https://www.metacritic.com/",
"username_claimed": "blue"
}
```

## G2G
__2024-04-10 :__ Seems to be loading profiles with some wierd javascript setup that sherlock doesn't like, leading to difficult to control false positives
```json
"G2G": {
"errorType": "response_url",
"errorUrl": "https://www.g2g.com/{}",
"regexCheck": "^[A-Za-z][A-Za-z0-9_]{2,11}$",
"url": "https://www.g2g.com/{}",
"urlMain": "https://www.g2g.com/",
"username_claimed": "user"
}
```

## Bitcoin Forum
__2024-04-24 :__ BCF seems to have gone defunct. Uncertain.
```json
"BitCoinForum": {
"errorMsg": "The user whose profile you are trying to view does not exist.",
"errorType": "message",
"url": "https://bitcoinforum.com/profile/{}",
"urlMain": "https://bitcoinforum.com",
"username_claimed": "bitcoinforum.com"
}
```
9 changes: 9 additions & 0 deletions sherlock/notify.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,15 @@ def update(self, result):
Fore.WHITE + "]" +
Fore.GREEN + f" {self.result.site_name}:" +
Fore.YELLOW + f" {msg}")

elif result.status == QueryStatus.WAF:
if self.print_all:
print(Style.BRIGHT + Fore.WHITE + "[" +
Fore.RED + "-" +
Fore.WHITE + "]" +
Fore.GREEN + f" {self.result.site_name}:" +
Fore.RED + " Blocked by bot detection" +
Fore.YELLOW + " (proxy may help)")

else:
# It should be impossible to ever get here...
Expand Down
Loading

0 comments on commit 54597f3

Please sign in to comment.