Merge pull request #46 from 1nchaos/dev

v1.2.0

HISTORY.md

@@ -9,9 +9,10 @@ master
 
 专注股票量化数据，为Ai(爱)发电，向阳而生。
 
-1.1.0 (2023-10-23)
+1.2.0 (2023-12-04)
 ------------------
-1. 修复部分已知bug。
+1. 新增：根据股票代码获取单只股票所属概念的接口。
+
 
 1.1.0 (2023-10-18)
 ------------------
@@ -33,7 +34,6 @@ master
 1. 优化：部分接口兼容pandas2.x问题。
 2. 优化：股票代码列表，新增近期新股部分。
 
-
 0.0.30b0 (2023-08-08)
 ------------------
 1. 优化：同花顺 cookie设置，保障同花顺的数据获取。

README.md

@@ -134,21 +134,23 @@ print(res_df)
 
 #### 	1. 基本信息
 
-| 数据             | API                                   | 说明                                   | 备注                                                      |
-| ---------------- | ------------------------------------- | -------------------------------------- | --------------------------------------------------------- |
-| A股代码          | stock.info.all_code()                 | 所有A股代码信息                        |                                                           |
-| **概念**         |                                       |                                        |                                                           |
-| 来源：同花顺     |                                       |                                        |                                                           |
-| 概念代码         | stock.info.all_concept_code_ths()     | 所有A股概念代码信息（同花顺）          | 来源：同花顺公开数据                                      |
-| 概念成分列表     | stock.info.concept_constituent_ths()  | 获取同花顺概念指数的成分股（同花顺）   | 注意：返回结果只有股票代码和股票简称，可根据概念名称查询  |
-| 来源：东方财富   |                                       |                                        |                                                           |
-| 概念代码         | stock.info.all_concept_code_east()    | 所有A股概念代码信息（东方财富）        | 来源：[东方财富](https://data.eastmoney.com/bkzj/gn.html) |
-| 概念成分列表     | stock.info.concept_constituent_east() | 获取同花顺概念指数的成分股（东方财富） | 注意：返回结果只有股票代码和股票简称，可根据概念名称查询  |
-| **指数**         |                                       |                                        |                                                           |
-| 指数代码         | stock.info.all_index_code()           | 获取所有A股市场的指数代码              | 来源同花顺，可能存在同花顺对代码重新编码的情况            |
-| 指数对应的成分股 | stock.info.index_constituent()        | 获取对应指数的成分股列表               |                                                           |
-| **其它**         |                                       |                                        |                                                           |
-| 股票交易日历     | stock.info.trade_calendar()           | 获取股票交易日信息                     | 来源：深交所                                              |
+| 数据             | API                                   | 说明                                   | 备注                                                         |
+| ---------------- | ------------------------------------- | -------------------------------------- | ------------------------------------------------------------ |
+| A股代码          | stock.info.all_code()                 | 所有A股代码信息                        |                                                              |
+| **概念**         |                                       |                                        |                                                              |
+| 来源：同花顺     |                                       |                                        |                                                              |
+| 概念代码         | stock.info.all_concept_code_ths()     | 所有A股概念代码信息（同花顺）          | 来源：同花顺公开数据                                         |
+| 概念成分列表     | stock.info.concept_constituent_ths()  | 获取同花顺概念指数的成分股（同花顺）   | 注意：返回结果只有股票代码和股票简称，可根据概念名称查询     |
+| 股票所属概念     | stock.info.get_concept_ths()          | 获取单只股票所属的概念板块             | [F10](https://basic.10jqka.com.cn/300033/concept.html)       |
+| 来源：东方财富   |                                       |                                        |                                                              |
+| 概念代码         | stock.info.all_concept_code_east()    | 所有A股概念代码信息（东方财富）        | 来源：[东方财富](https://data.eastmoney.com/bkzj/gn.html)    |
+| 概念成分列表     | stock.info.concept_constituent_east() | 获取同花顺概念指数的成分股（东方财富） | 注意：返回结果只有股票代码和股票简称，可根据概念名称查询     |
+| 股票所属概念     | stock.info.get_concept_east()         | 获取单只股票所属的概念板块             | [核心题材](https://emweb.securities.eastmoney.com/pc_hsf10/pages/index.html?type=web&code=SZ300059&color=b#/hxtc) |
+| **指数**         |                                       |                                        |                                                              |
+| 指数代码         | stock.info.all_index_code()           | 获取所有A股市场的指数代码              | 来源同花顺，可能存在同花顺对代码重新编码的情况               |
+| 指数对应的成分股 | stock.info.index_constituent()        | 获取对应指数的成分股列表               |                                                              |
+| **其它**         |                                       |                                        |                                                              |
+| 股票交易日历     | stock.info.trade_calendar()           | 获取股票交易日信息                     | 来源：深交所                                                 |
 
 #### 	2. 行情信息
 
@@ -252,8 +254,8 @@ print(res_df)
 
 > 对于项目有支持，包括但不仅限：内容贡献，bug提交，思想交流等等，对项目有影响的个人和机构
 
-| Simon | [bigbigbigfish](https://github.com/bigbigbigfish) | [LuneZ99](https://github.com/LuneZ99) | 匿名用户 | thue | [Triones009](https://github.com/Triones009) | [lzd-1230](https://github.com/lzd-1230) |      |
-| ----- | ------------------------------------------------- | ------------------------------------- | -------- | ---- | ------------------------------------------- | --------------------------------------- | ---- |
+| Simon | [bigbigbigfish](https://github.com/bigbigbigfish) | [LuneZ99](https://github.com/LuneZ99) | 匿名用户 | thue | [Triones009](https://github.com/Triones009) | [lzd-1230](https://github.com/lzd-1230) | [hanxuanliang](https://github.com/hanxuanliang) |
+| ----- | ------------------------------------------------- | ------------------------------------- | -------- | ---- | ------------------------------------------- | --------------------------------------- | ----------------------------------------------- |
 
 ----------------------------------------------------------------------
 

adata/common/utils/code_utils.py

@@ -0,0 +1,23 @@
+# -*- coding: utf-8 -*-
+"""
+@desc: readme
+@author: yinchao
+@time: 2023/12/4
+@log: change log
+"""
+
+
+def compile_exchange_by_stock_code(stock_code):
+    """根据股票代码补全市场后缀"""
+    exchange_suffix = {
+        '0': '.SZ',
+        '3': '.SZ',
+        '6': '.SH',
+        '9': '.SH',
+        '4': '.BJ',
+        '8': '.BJ'
+    }
+    prefix = stock_code[0]
+    if prefix in exchange_suffix:
+        return stock_code + exchange_suffix[prefix]
+    return stock_code

adata/stock/info/concept/stock_concept_east.py

@@ -4,13 +4,17 @@
 东方财富股票概念
 
 https://data.eastmoney.com/bkzj/gn.html
+
+单个股票的所有概念板块
+https://datacenter.eastmoney.com/securities/api/data/v1/get?reportName=RPT_F10_CORETHEME_BOARDTYPE&columns=SECUCODE%2CSECURITY_CODE%2CSECURITY_NAME_ABBR%2CNEW_BOARD_CODE%2CBOARD_NAME%2CSELECTED_BOARD_REASON%2CIS_PRECISE%2CBOARD_RANK%2CBOARD_YIELD%2CDERIVE_BOARD_CODE&quoteColumns=f3~05~NEW_BOARD_CODE~BOARD_YIELD&filter=(SECUCODE%3D%22600138.SH%22)(IS_PRECISE%3D%221%22)&pageNumber=1&pageSize=&sortTypes=1&sortColumns=BOARD_RANK&source=HSF10&client=PC&v=0029565688091059528
 @author: 1nchaos
 @date: 2023/3/30 16:17
 """
 
 import pandas as pd
 
 from adata.common import requests
+from adata.common.utils.code_utils import compile_exchange_by_stock_code
 from adata.stock.info.concept.stock_concept_template import StockConceptTemplate
 
 
@@ -61,7 +65,38 @@ def concept_constituent_east(self, concept_code=None, wait_time=None):
         result_df = pd.DataFrame(data=data, columns=self._CONCEPT_CONSTITUENT_COLUMNS)
         return result_df
 
+    def get_concept_east(self, stock_code: str = '000001'):
+        """
+        根据股票代码获取，股票所属的所有的概念信息
+        https://datacenter.eastmoney.com/securities/api/data/v1/get?
+        reportName=RPT_F10_CORETHEME_BOARDTYPE
+        &columns=SECUCODE%2CSECURITY_CODE%2CSECURITY_NAME_ABBR%2CNEW_BOARD_CODE%2CBOARD_NAME%2CSELECTED_BOARD_REASON%2CIS_PRECISE%2CBOARD_RANK%2CBOARD_YIELD%2CDERIVE_BOARD_CODE
+        &quoteColumns=f3~05~NEW_BOARD_CODE~BOARD_YIELD
+        &filter=(SECUCODE%3D%22600138.SH%22)(IS_PRECISE%3D%221%22)
+        &pageNumber=1&pageSize=&sortTypes=1&sortColumns=BOARD_RANK&source=HSF10&client=PC&v=0029565688091059528
+        :param stock_code: 股票代码
+        :return: 概念信息
+        """
+        stock_code = compile_exchange_by_stock_code(stock_code)
+        url = f"https://datacenter.eastmoney.com/securities/api/data/v1/get?" \
+              f"reportName=RPT_F10_CORETHEME_BOARDTYPE&" \
+              f"columns=SECUCODE%2CSECURITY_CODE%2CSECURITY_NAME_ABBR%2CNEW_BOARD_CODE%2CBOARD_NAME%2CSELECTED_BOARD_REASON%2CIS_PRECISE%2CBOARD_RANK%2CBOARD_YIELD%2CDERIVE_BOARD_CODE&" \
+              f"quoteColumns=f3~05~NEW_BOARD_CODE~BOARD_YIELD&" \
+              f"filter=(SECUCODE%3D%22{stock_code}%22)(IS_PRECISE%3D%221%22)&pageNumber=1&pageSize=50&sortTypes=1&" \
+              f"sortColumns=BOARD_RANK&source=HSF10&client=PC"
+        res_json = requests.request('get', url, headers={}, proxies={}).json()
+        res_data = res_json['result']['data']
+        data = []
+        for _ in res_data:
+            # ['stock_code', 'short_name', 'concept_code', 'name', 'reason', 'source']
+            data.append({'stock_code': _['SECURITY_CODE'], 'concept_code': _['NEW_BOARD_CODE'],
+                         'name': _['BOARD_NAME'],
+                         'reason': _['SELECTED_BOARD_REASON'], 'source': '东方财富'})
+        result_df = pd.DataFrame(data=data, columns=self._CONCEPT_INFO_COLUMNS)
+        return result_df
+
 
 if __name__ == '__main__':
     print(StockConceptEast().all_concept_code_east())
     print(StockConceptEast().concept_constituent_east(concept_code="BK0637"))
+    print(StockConceptEast().get_concept_east(stock_code="600020").to_string())

adata/stock/info/concept/stock_concept_template.py

@@ -13,6 +13,7 @@ class StockConceptTemplate(object):
     """
     _CONCEPT_CONSTITUENT_COLUMNS = ['stock_code', 'short_name']
     _CONCEPT_CODE_COLUMNS = ['concept_code', 'index_code', 'name', 'source']
+    _CONCEPT_INFO_COLUMNS = ['stock_code', 'concept_code', 'name', 'source', 'reason']
 
     def __init__(self) -> None:
         super().__init__()

adata/stock/info/concept/stock_concept_ths.py

@@ -6,6 +6,8 @@
 概念，指数成分
 来源于同花顺
 http://q.10jqka.com.cn/gn
+
+https://basic.10jqka.com.cn/000002/concept.html?cid=308717#ifind
 @author: 1nchaos
 @date: 2023/3/30 16:17
 """
@@ -255,6 +257,34 @@ def __index_constituent_ths_by_name(self, name=None, wait_time=None):
         data.clear()
         return result_df[self._CONCEPT_CONSTITUENT_COLUMNS]
 
+    def get_concept_ths(self, stock_code: str = '000001'):
+        """
+        根据股票代码获取，股票所属的所有的概念信息
+        https://basic.10jqka.com.cn/300033/concept.html
+        :param stock_code: 股票代码
+        :return: 概念信息
+        """
+        url = f"https://basic.10jqka.com.cn/{stock_code}/concept.html"
+        headers = ths_headers.text_headers
+        headers['Host'] = 'basic.10jqka.com.cn'
+        res = requests.request('get', url, headers=headers, proxies={})
+        # 3. 解析数据
+        text = res.content.decode('gbk')
+        soup = BeautifulSoup(text, 'html.parser')
+        table = soup.find('table', attrs={'class': 'gnContent'})
+        trs = table.tbody.find_all('tr')
+        data = []
+        for i in range(0, len(trs), 2):
+            columns = trs[i].find_all('td')
+            data.append({'stock_code': stock_code, 'concept_code': columns[1].get('clid'),
+                         'name': columns[1].text,
+                         'reason': trs[i + 1].text, 'source': '同花顺'})
+        result_df = pd.DataFrame(data=data, columns=self._CONCEPT_INFO_COLUMNS)
+        result_df.replace(to_replace=[r'\t', r'\n', ' '], value='', regex=True, inplace=True)
+        return result_df
+
 
 if __name__ == '__main__':
-    print(StockConceptThs().all_concept_code_ths())
+    # print(StockConceptThs().all_concept_code_ths())
+    print(StockConceptThs().get_concept_ths(stock_code='300033')[
+              ['stock_code', 'concept_code', 'name', 'source','reason']].to_string())

adata/stock/market/stock_market/stock_market_baidu.py

@@ -77,6 +77,8 @@ def get_market(self, stock_code: str = '000001', start_date='1990-01-01', k_type
         rename_columns = {'turnoverratio': 'turnover_ratio', 'preClose': 'pre_close', 'range': 'change',
                           'ratio': 'change_pct', 'time': 'trade_time'}
         result_df = pd.DataFrame(data=data, columns=keys).rename(columns=rename_columns)[self._MARKET_COLUMNS]
+        if result_df.empty:
+            return pd.DataFrame(data=[], columns=self._MARKET_COLUMNS)
         result_df['stock_code'] = stock_code
         result_df['trade_date'] = result_df['trade_time']
         result_df['trade_time'] = pd.to_datetime(result_df['trade_time']).dt.strftime('%Y-%m-%d %H:%M:%S')

requirements.txt

@@ -1,4 +1,4 @@
 requests>=2.16.0
-pandas>=1.1.5
+pandas>=0.22.0
 beautifulsoup4>=4.0.2
 py_mini_racer>=0.6.0

setup.py

@@ -19,7 +19,7 @@
 
 requires = [
     "requests>=2.16.0",
-    "pandas>=1.1.5",
+    "pandas>=0.22.0",
     "beautifulsoup4>=4.0.2",
     "py_mini_racer>=0.6.0",
 ]

tests/adata_test/stock/info_test.py

@@ -44,6 +44,12 @@ def test_concept_constituent_ths(self):
         print(df)
         self.assertEqual(True, len(df) > 200)
 
+    def test_get_concept_ths(self):
+        print("开始测试：get_concept_ths")
+        df = adata.stock.info.get_concept_ths(stock_code="000002")
+        print(df)
+        self.assertEqual(True, len(df) > 6)
+
     def test_all_concept_code_east(self):
         print("开始测试：test_all_concept_code_ths")
         df = adata.stock.info.all_concept_code_east()
@@ -56,6 +62,12 @@ def test_concept_constituent_east(self):
         print(df)
         self.assertEqual(True, len(df) > 200)
 
+    def test_get_concept_east(self):
+        print("开始测试：get_concept_ths")
+        df = adata.stock.info.get_concept_east(stock_code="000002")
+        print(df)
+        self.assertEqual(True, len(df) > 6)
+
     def test_all_index_code(self):
         print("开始测试：test_all_index_code")
         df = adata.stock.info.all_index_code()

tests/utils/csv_utils.py

@@ -12,7 +12,7 @@
 
 def calendar_csv():
     for i in range(20):
-        year = 2013 - i
+        year = 2024 - i
         df = adata.stock.info.trade_calendar(year=year)
         df.to_csv(f'{year}.csv', index=False)
 
@@ -28,4 +28,4 @@ def code_csv():
 
 
 if __name__ == '__main__':
-    code_csv()
+    calendar_csv()